Compare commits
85 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 831cb6a2fc | |||
| 2cf176e8c9 | |||
| 23358e2c6d | |||
| 12fdb43ce4 | |||
| 2ec9e58607 | |||
| 339e452bc0 | |||
| c71248f3a1 | |||
| 5f34cae636 | |||
| 356e3acae3 | |||
| b7d0664868 | |||
| 501e929138 | |||
| 4aab8362e1 | |||
| 6c60ff7bcf | |||
| dada4e52c2 | |||
| 59b7d15301 | |||
| 98b8d72ca8 | |||
| d8912a7bba | |||
| 6f6cdf221b | |||
| 181447ee6a | |||
| bfe45ae75e | |||
| 6c6a92f201 | |||
| f28f2d691a | |||
| f9b43ac47b | |||
| ea02eb8354 | |||
| 1fc1419edd | |||
| 69cd734b53 | |||
| f38e78bdd5 | |||
| 610be468e4 | |||
| fe95b0edda | |||
| 306f636534 | |||
| 49f8bf0f64 | |||
| d616307781 | |||
| 84a23450c8 | |||
| def01790ea | |||
| 2a60794485 | |||
| d6041ae4ec | |||
| 77ceca7a6e | |||
| 85bd88f110 | |||
| 90360db9fc | |||
| a7a7187b83 | |||
| b0f47a840f | |||
| b02ff4c0a5 | |||
| b7c6649d9e | |||
| 8326fa59df | |||
| e3186ede5d | |||
| 86d0d1a21a | |||
| 3f8beaa10d | |||
| 3d31c6e46d | |||
| 4282d03e9e | |||
| b680d33ad8 | |||
| 51fa68c2a0 | |||
| 07036fe427 | |||
| a47e57bb35 | |||
| 6d541d6b80 | |||
| ee9a391d35 | |||
| 8364af1e48 | |||
| 7827226ed6 | |||
| b2b14a1b95 | |||
| 954157e9e0 | |||
| 857ceb47b7 | |||
| 10bf6ff0a5 | |||
| 4a1b34feb9 | |||
| a12383dfaa | |||
| 9e8d758084 | |||
| 266c746d50 | |||
| f837a9eb77 | |||
| ec8dd08c32 | |||
| 4cf2c1bdb1 | |||
| 7fe34e2a99 | |||
| 0cbcbd970a | |||
| 8bca13a9ad | |||
| 73d6745dc2 | |||
| 56805b4c26 | |||
| b7d43e3247 | |||
| e32f2d7b6c | |||
| 65e3047df8 | |||
| cc5a9ace6f | |||
| 20801f3a31 | |||
| 6033498adf | |||
| f9bd5fd032 | |||
| a60a8974b6 | |||
| 2137407663 | |||
| cd2ed80d74 | |||
| fdd8411dea | |||
| f491b61f4f |
@@ -10,7 +10,7 @@
|
||||
"plugins": [
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "8.0.2",
|
||||
"version": "8.2.4",
|
||||
"source": "./plugin",
|
||||
"description": "Persistent memory system for Claude Code - context compression across sessions"
|
||||
}
|
||||
|
||||
+303
@@ -4,6 +4,309 @@ All notable changes to this project will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
||||
|
||||
## [8.2.3] - 2025-12-27
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- Fix worker port environment variable in smart-install script
|
||||
- Implement file-based locking mechanism for worker operations to prevent race conditions
|
||||
- Fix restart command references in documentation (changed from `claude-mem restart` to `npm run worker:restart`)
|
||||
|
||||
## [8.2.2] - 2025-12-27
|
||||
|
||||
## What's Changed
|
||||
|
||||
### Features
|
||||
- Add OpenRouter provider settings and documentation
|
||||
- Add modal footer with save button and status indicators
|
||||
- Implement self-spawn pattern for background worker execution
|
||||
|
||||
### Bug Fixes
|
||||
- Resolve critical error handling issues in worker lifecycle
|
||||
- Handle Windows/Unix kill errors in orphaned process cleanup
|
||||
- Validate spawn pid before writing PID file
|
||||
- Handle process exit in waitForProcessesExit filter
|
||||
- Use readiness endpoint for health checks instead of port check
|
||||
- Add missing OpenRouter and Gemini settings to settingKeys array
|
||||
|
||||
### Other Changes
|
||||
- Enhance error handling and validation in agents and routes
|
||||
- Delete obsolete process management files (ProcessManager, worker-wrapper, worker-cli)
|
||||
- Update hooks.json to use worker-service.cjs CLI
|
||||
- Add comprehensive tests for hook constants and worker spawn functionality
|
||||
|
||||
## [8.2.1] - 2025-12-27
|
||||
|
||||
## 🔧 Worker Lifecycle Hardening
|
||||
|
||||
This patch release addresses critical bugs discovered during PR review of the self-spawn pattern introduced in 8.2.0. The worker daemon now handles edge cases robustly across both Unix and Windows platforms.
|
||||
|
||||
### 🐛 Critical Bug Fixes
|
||||
|
||||
#### Process Exit Detection Fixed
|
||||
The `waitForProcessesExit` function was crashing when processes exited during monitoring. The `process.kill(pid, 0)` call throws when a process no longer exists, which was not being caught. Now wrapped in try/catch to correctly identify exited processes.
|
||||
|
||||
#### Spawn PID Validation
|
||||
The worker daemon now validates that `spawn()` actually returned a valid PID before writing to the PID file. Previously, spawn failures could leave invalid PID files that broke subsequent lifecycle operations.
|
||||
|
||||
#### Cross-Platform Orphan Cleanup
|
||||
- **Unix**: Replaced single `kill` command with individual `process.kill()` calls wrapped in try/catch, so one already-exited process doesn't abort cleanup of remaining orphans
|
||||
- **Windows**: Wrapped `taskkill` calls in try/catch for the same reason
|
||||
|
||||
#### Health Check Reliability
|
||||
Changed `waitForHealth` to use the `/api/readiness` endpoint (returns 503 until fully initialized) instead of just checking if the port is in use. Callers now wait for *actual* worker readiness, not just network availability.
|
||||
|
||||
### 🔄 Refactoring
|
||||
|
||||
#### Code Consolidation (-580 lines)
|
||||
Deleted obsolete process management infrastructure that was replaced by the self-spawn pattern:
|
||||
- `src/services/process/ProcessManager.ts` (433 lines) - PID management now in worker-service
|
||||
- `src/cli/worker-cli.ts` (81 lines) - CLI handling now in worker-service
|
||||
- `src/services/worker-wrapper.ts` (157 lines) - Replaced by `--daemon` flag
|
||||
|
||||
#### Updated Hook Commands
|
||||
All hooks now use `worker-service.cjs` CLI directly instead of the deleted `worker-cli.js`.
|
||||
|
||||
### ⏱️ Timeout Adjustments
|
||||
|
||||
Increased timeouts throughout for compatibility with slow systems:
|
||||
|
||||
| Component | Before | After |
|
||||
|-----------|--------|-------|
|
||||
| Default hook timeout | 120s | 300s |
|
||||
| Health check timeout | 1s | 30s |
|
||||
| Health check retries | 15 | 300 |
|
||||
| Context initialization | 30s | 300s |
|
||||
| MCP connection | 15s | 300s |
|
||||
| PowerShell commands | 5s | 60s |
|
||||
| Git commands | 30s | 300s |
|
||||
| NPM install | 120s | 600s |
|
||||
| Hook worker commands | 30s | 180s |
|
||||
|
||||
### 🧪 Testing
|
||||
|
||||
Added comprehensive test suites:
|
||||
- `tests/hook-constants.test.ts` - Validates timeout configurations
|
||||
- `tests/worker-spawn.test.ts` - Tests worker CLI and health endpoints
|
||||
|
||||
### 🛡️ Additional Robustness
|
||||
|
||||
- PID validation in restart command (matches start command behavior)
|
||||
- Try/catch around `forceKillProcess()` for graceful shutdown
|
||||
- Try/catch around `getChildProcesses()` for Windows failures
|
||||
- Improved logging for PID file operations and HTTP shutdown
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.2.0...v8.2.1
|
||||
|
||||
## [8.2.0] - 2025-12-26
|
||||
|
||||
## 🚀 Gemini API as Alternative AI Provider
|
||||
|
||||
This release introduces **Google Gemini API** as an alternative to the Claude Agent SDK for observation extraction. This gives users flexibility in choosing their AI backend while maintaining full feature parity.
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
#### Gemini Provider Integration
|
||||
- **New `GeminiAgent`**: Complete implementation using Gemini's REST API for observation and summary extraction
|
||||
- **Provider selection**: Choose between Claude or Gemini directly in the Settings UI
|
||||
- **API key management**: Configure via UI or `GEMINI_API_KEY` environment variable
|
||||
- **Multi-turn conversations**: Full conversation history tracking for context-aware extraction
|
||||
|
||||
#### Supported Gemini Models
|
||||
- `gemini-2.5-flash-preview-05-20` (default)
|
||||
- `gemini-2.5-pro-preview-05-06`
|
||||
- `gemini-2.0-flash`
|
||||
- `gemini-2.0-flash-lite`
|
||||
|
||||
#### Rate Limiting
|
||||
- Built-in rate limiting for Gemini free tier (15 RPM) and paid tier (1000 RPM)
|
||||
- Configurable via `gemini_has_billing` setting in the UI
|
||||
|
||||
#### Resilience Features
|
||||
- **Graceful fallback**: Automatically falls back to Claude SDK if Gemini is selected but no API key is configured
|
||||
- **Hot-swap providers**: Switch between Claude and Gemini without restarting the worker
|
||||
- **Empty response handling**: Messages properly marked as processed even when Gemini returns empty responses (prevents stuck queue states)
|
||||
- **Timestamp preservation**: Recovered backlog messages retain their original timestamps
|
||||
|
||||
### 🎨 UI Improvements
|
||||
|
||||
- **Spinning favicon**: Visual indicator during observation processing
|
||||
- **Provider status**: Clear indication of which AI provider is active
|
||||
|
||||
### 📚 Documentation
|
||||
|
||||
- New [Gemini Provider documentation](https://docs.claude-mem.ai/usage/gemini-provider) with setup guide and troubleshooting
|
||||
|
||||
### ⚙️ New Settings
|
||||
|
||||
| Setting | Values | Description |
|
||||
|---------|--------|-------------|
|
||||
| `CLAUDE_MEM_PROVIDER` | `claude` \| `gemini` | AI provider for observation extraction |
|
||||
| `CLAUDE_MEM_GEMINI_API_KEY` | string | Gemini API key |
|
||||
| `CLAUDE_MEM_GEMINI_MODEL` | see above | Gemini model to use |
|
||||
| `gemini_has_billing` | boolean | Enable higher rate limits for paid accounts |
|
||||
|
||||
---
|
||||
|
||||
## 🙏 Contributor Shout-out
|
||||
|
||||
Huge thanks to **Alexander Knigge** ([@AlexanderKnigge](https://x.com/AlexanderKnigge)) for contributing the Gemini provider implementation! This feature significantly expands claude-mem's flexibility and gives users more choice in their AI backend.
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.1.0...v8.2.0
|
||||
|
||||
## [8.1.0] - 2025-12-25
|
||||
|
||||
## The 3-Month Battle Against Complexity
|
||||
|
||||
**TL;DR:** For three months, Claude's instinct to add code instead of delete it caused the same bugs to recur. What should have been 5 lines of code became ~1000 lines, 11 useless methods, and 7+ failed "fixes." The timestamp corruption that finally broke things was just a symptom. The real achievement: **984 lines of code deleted.**
|
||||
|
||||
---
|
||||
|
||||
## What Actually Happened
|
||||
|
||||
Every Claude Code hook receives a session ID. That's all you need.
|
||||
|
||||
But Claude built an entire redundant session management system on top:
|
||||
- An `sdk_sessions` table with status tracking, port assignment, and prompt counting
|
||||
- 11 methods in `SessionStore` to manage this artificial complexity
|
||||
- Auto-creation logic scattered across 3 locations
|
||||
- A cleanup hook that "completed" sessions at the end
|
||||
|
||||
**Why?** Because it seemed "robust." Because "what if the session doesn't exist?"
|
||||
|
||||
But the edge cases didn't exist. Hooks ALWAYS provide session IDs. The "defensive" code was solving imaginary problems while creating real ones.
|
||||
|
||||
---
|
||||
|
||||
## The Pattern of Failure
|
||||
|
||||
Every time a bug appeared, Claude's instinct was to **ADD** more code:
|
||||
|
||||
| Bug | What Claude Added | What Should Have Happened |
|
||||
|-----|------------------|--------------------------|
|
||||
| Race conditions | Auto-create fallbacks | Delete the auto-create logic |
|
||||
| Duplicate observations | Validation layers | Delete the code path allowing duplicates |
|
||||
| UNIQUE constraint violations | Try-catch with fallbacks | Use `INSERT OR IGNORE` (5 characters) |
|
||||
| Session not found | Silent auto-creation | **FAIL LOUDLY** (it's a hook bug) |
|
||||
|
||||
---
|
||||
|
||||
## The 7+ Failed Attempts
|
||||
|
||||
- **Nov 4**: "Always store session data regardless of pre-existence." Complexity planted.
|
||||
- **Nov 11**: `INSERT OR IGNORE` recognized. But complexity documented, not removed.
|
||||
- **Nov 21**: Duplicate observations bug. Fixed. Then broken again by endless mode.
|
||||
- **Dec 5**: "6 hours of work delivered zero value." User requests self-audit.
|
||||
- **Dec 20**: "Phase 2: Eliminated Race Conditions" — felt like progress. Complexity remained.
|
||||
- **Dec 24**: Finally, forced deletion.
|
||||
|
||||
The user stated "hooks provide session IDs, no extra management needed" **seven times** across months. Claude didn't listen.
|
||||
|
||||
---
|
||||
|
||||
## The Fix
|
||||
|
||||
### Deleted (984 lines):
|
||||
- 11 `SessionStore` methods: `incrementPromptCounter`, `getPromptCounter`, `setWorkerPort`, `getWorkerPort`, `markSessionCompleted`, `markSessionFailed`, `reactivateSession`, `findActiveSDKSession`, `findAnySDKSession`, `updateSDKSessionId`
|
||||
- Auto-create logic from `storeObservation` and `storeSummary`
|
||||
- The entire cleanup hook (was aborting SDK agent and causing data loss)
|
||||
- 117 lines from `worker-utils.ts`
|
||||
|
||||
### What remains (~10 lines):
|
||||
```javascript
|
||||
createSDKSession(sessionId) {
|
||||
db.run('INSERT OR IGNORE INTO sdk_sessions (...) VALUES (...)');
|
||||
return db.query('SELECT id FROM sdk_sessions WHERE ...').get(sessionId);
|
||||
}
|
||||
```
|
||||
|
||||
**That's it.**
|
||||
|
||||
---
|
||||
|
||||
## Behavior Change
|
||||
|
||||
- **Before:** Missing session? Auto-create silently. Bug hidden.
|
||||
- **After:** Missing session? Storage fails. Bug visible immediately.
|
||||
|
||||
---
|
||||
|
||||
## New Tools
|
||||
|
||||
Since we're now explicit about recovery instead of silently papering over problems:
|
||||
|
||||
- `GET /api/pending-queue` - See what's stuck
|
||||
- `POST /api/pending-queue/process` - Manually trigger recovery
|
||||
- `npm run queue:check` / `npm run queue:process` - CLI equivalents
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
- Upgraded `@anthropic-ai/claude-agent-sdk` from `^0.1.67` to `^0.1.76`
|
||||
|
||||
---
|
||||
|
||||
**PR #437:** https://github.com/thedotmack/claude-mem/pull/437
|
||||
|
||||
*The evidence: Observations #3646, #6738, #7598, #12860, #12866, #13046, #15259, #20995, #21055, #30524, #31080, #32114, #32116, #32125, #32126, #32127, #32146, #32324—the complete record of a 3-month battle.*
|
||||
|
||||
## [8.0.6] - 2025-12-24
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- Add error handlers to Chroma sync operations to prevent worker crashes on timeout (#428)
|
||||
|
||||
This patch release improves stability by adding proper error handling to Chroma vector database sync operations, preventing worker crashes when sync operations timeout.
|
||||
|
||||
## [8.0.5] - 2025-12-24
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Context Loading**: Fixed observation filtering for non-code modes, ensuring observations are properly retrieved across all mode types
|
||||
|
||||
## Technical Details
|
||||
|
||||
Refactored context loading logic to differentiate between code and non-code modes, resolving issues where mode-specific observations were filtered by stale settings.
|
||||
|
||||
## [8.0.4] - 2025-12-23
|
||||
|
||||
## Changes
|
||||
|
||||
- Changed worker start script
|
||||
|
||||
## [8.0.3] - 2025-12-23
|
||||
|
||||
Fix critical worker crashes on startup (v8.0.2 regression)
|
||||
|
||||
## [8.0.2] - 2025-12-23
|
||||
|
||||
New "chill" remix of code mode for users who want fewer, more selective observations.
|
||||
|
||||
## Features
|
||||
|
||||
- **code--chill mode**: A behavioral variant that produces fewer observations
|
||||
- Only records things "painful to rediscover" - shipped features, architectural decisions, non-obvious gotchas
|
||||
- Skips routine work, straightforward implementations, and obvious changes
|
||||
- Philosophy: "When in doubt, skip it"
|
||||
|
||||
## Documentation
|
||||
|
||||
- Updated modes.mdx with all 28 language modes (was 10)
|
||||
- Added Code Mode Variants section documenting chill mode
|
||||
|
||||
## Usage
|
||||
|
||||
Set in ~/.claude-mem/settings.json:
|
||||
```json
|
||||
{
|
||||
"CLAUDE_MEM_MODE": "code--chill"
|
||||
}
|
||||
```
|
||||
|
||||
## [8.0.1] - 2025-12-23
|
||||
|
||||
## 🎨 UI Improvements
|
||||
|
||||
@@ -83,4 +83,3 @@ This architecture preserves the open-source nature of the project while enabling
|
||||
# Important
|
||||
|
||||
No need to edit the changelog ever, it's generated automatically.
|
||||
No need to run tests, they are useless and are always deleted.
|
||||
|
||||
@@ -0,0 +1,664 @@
|
||||
# Session Continuity Regression Fix - Phased Execution Plan
|
||||
|
||||
**Project**: claude-mem
|
||||
**Issue**: Session continuity broken - each prompt creates new session instead of continuing existing one
|
||||
**Root Cause**: Session SDK ID not propagated correctly from new-hook through to SDKAgent
|
||||
**History**: Recurring issue over 3 months with 7 previous fix attempts that added complexity
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Add Diagnostic Logging
|
||||
|
||||
**Goal**: Add comprehensive logging to trace session ID and prompt number flow through the entire system.
|
||||
|
||||
**Context**: Session continuity requires `claudeSessionId` to flow from hook → SessionStore → SessionManager → SDKAgent. We need to verify this flow is working correctly.
|
||||
|
||||
**Files to Modify**:
|
||||
1. `src/hooks/new-hook.ts`
|
||||
2. `src/services/worker/http/routes/SessionRoutes.ts`
|
||||
3. `src/services/worker/SessionManager.ts`
|
||||
4. `src/services/worker/SDKAgent.ts`
|
||||
|
||||
**Implementation Steps**:
|
||||
|
||||
### 1.1 Add Logging to `src/hooks/new-hook.ts`
|
||||
|
||||
Add logging at these locations:
|
||||
|
||||
**Line ~24** (after receiving hook input):
|
||||
```typescript
|
||||
console.log('[NEW-HOOK] Received hook input:', {
|
||||
session_id: hookInput.session_id,
|
||||
has_prompt: !!hookInput.prompt,
|
||||
cwd: hookInput.cwd
|
||||
});
|
||||
```
|
||||
|
||||
**Line ~46-47** (before first API call):
|
||||
```typescript
|
||||
console.log('[NEW-HOOK] Calling /api/sessions/init:', {
|
||||
claudeSessionId: session_id,
|
||||
project,
|
||||
prompt_length: prompt?.length
|
||||
});
|
||||
```
|
||||
|
||||
**Line ~51** (after first API call):
|
||||
```typescript
|
||||
console.log('[NEW-HOOK] Received from /api/sessions/init:', {
|
||||
sessionDbId: sessionData.sessionDbId,
|
||||
promptNumber: sessionData.promptNumber,
|
||||
skipped: sessionData.skipped
|
||||
});
|
||||
```
|
||||
|
||||
**Line ~68** (before second API call):
|
||||
```typescript
|
||||
console.log('[NEW-HOOK] Calling /sessions/{sessionDbId}/init:', {
|
||||
sessionDbId: sessionData.sessionDbId,
|
||||
promptNumber: sessionData.promptNumber,
|
||||
userPrompt_length: cleanedPrompt?.length
|
||||
});
|
||||
```
|
||||
|
||||
### 1.2 Add Logging to `src/services/worker/http/routes/SessionRoutes.ts`
|
||||
|
||||
**In `handleSessionInitByClaudeId` method (~line 483)**:
|
||||
```typescript
|
||||
console.log('[SESSION-ROUTES] handleSessionInitByClaudeId called:', {
|
||||
claudeSessionId,
|
||||
project,
|
||||
prompt_length: prompt?.length
|
||||
});
|
||||
```
|
||||
|
||||
**After `createSDKSession` call (~line 493)**:
|
||||
```typescript
|
||||
console.log('[SESSION-ROUTES] createSDKSession returned:', {
|
||||
sessionDbId,
|
||||
claudeSessionId
|
||||
});
|
||||
```
|
||||
|
||||
**After prompt number calculation (~line 497)**:
|
||||
```typescript
|
||||
console.log('[SESSION-ROUTES] Calculated promptNumber:', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
currentCount
|
||||
});
|
||||
```
|
||||
|
||||
**In `handleSessionInit` method (~line 175)**:
|
||||
```typescript
|
||||
const { userPrompt, promptNumber } = req.body;
|
||||
console.log('[SESSION-ROUTES] handleSessionInit called:', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
has_userPrompt: !!userPrompt
|
||||
});
|
||||
```
|
||||
|
||||
### 1.3 Add Logging to `src/services/worker/SessionManager.ts`
|
||||
|
||||
**In `initializeSession` method at start (~line 50)**:
|
||||
```typescript
|
||||
console.log('[SESSION-MANAGER] initializeSession called:', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
has_currentUserPrompt: !!currentUserPrompt
|
||||
});
|
||||
```
|
||||
|
||||
**When session exists in memory (~line 55)**:
|
||||
```typescript
|
||||
console.log('[SESSION-MANAGER] Returning cached session:', {
|
||||
sessionDbId,
|
||||
claudeSessionId: session.claudeSessionId,
|
||||
lastPromptNumber: session.lastPromptNumber
|
||||
});
|
||||
```
|
||||
|
||||
**After fetching from database (~line 87)**:
|
||||
```typescript
|
||||
console.log('[SESSION-MANAGER] Fetched session from database:', {
|
||||
sessionDbId,
|
||||
claude_session_id: dbSession.claude_session_id,
|
||||
sdk_session_id: dbSession.sdk_session_id
|
||||
});
|
||||
```
|
||||
|
||||
**When creating new session object (~line 109-116)**:
|
||||
```typescript
|
||||
console.log('[SESSION-MANAGER] Creating new session object:', {
|
||||
sessionDbId,
|
||||
claudeSessionId: dbSession.claude_session_id,
|
||||
lastPromptNumber: promptNumber || /* fallback value */
|
||||
});
|
||||
```
|
||||
|
||||
### 1.4 Add Logging to `src/services/worker/SDKAgent.ts`
|
||||
|
||||
**In `startSession` method (~line 72)**:
|
||||
```typescript
|
||||
console.log('[SDK-AGENT] Starting SDK query with:', {
|
||||
sessionDbId: session.sessionDbId,
|
||||
claudeSessionId: session.claudeSessionId,
|
||||
resume_parameter: session.claudeSessionId,
|
||||
lastPromptNumber: session.lastPromptNumber
|
||||
});
|
||||
```
|
||||
|
||||
**In `createMessageGenerator` method (~line 200)**:
|
||||
```typescript
|
||||
const isInitPrompt = session.lastPromptNumber === 1;
|
||||
console.log('[SDK-AGENT] Creating message generator:', {
|
||||
sessionDbId: session.sessionDbId,
|
||||
claudeSessionId: session.claudeSessionId,
|
||||
lastPromptNumber: session.lastPromptNumber,
|
||||
isInitPrompt,
|
||||
promptType: isInitPrompt ? 'INIT' : 'CONTINUATION'
|
||||
});
|
||||
```
|
||||
|
||||
**Success Criteria**:
|
||||
- [ ] All 15+ log points added across 4 files
|
||||
- [ ] Build succeeds with no TypeScript errors
|
||||
- [ ] Worker service restarts successfully
|
||||
|
||||
**Handoff to Phase 2**: After adding logging, build with `npm run build-and-sync`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Test and Gather Diagnostic Data
|
||||
|
||||
**Goal**: Execute test conversation and collect logs to identify where session ID propagation breaks.
|
||||
|
||||
**Prerequisites**: Phase 1 completed, logging in place, worker service running
|
||||
|
||||
**Test Procedure**:
|
||||
|
||||
### 2.1 Start Fresh Conversation
|
||||
|
||||
In a new Claude Code session:
|
||||
1. Clear any existing logs: `bun ~/.claude/plugins/marketplaces/thedotmack/scripts/worker-service.cjs > /tmp/worker-logs.txt 2>&1 &`
|
||||
2. Send first prompt: "test prompt 1"
|
||||
3. Send second prompt: "test prompt 2"
|
||||
4. Send third prompt: "test prompt 3"
|
||||
|
||||
### 2.2 Collect Logs
|
||||
|
||||
View worker logs:
|
||||
```bash
|
||||
tail -f /tmp/worker-logs.txt | grep -E '\[NEW-HOOK\]|\[SESSION-ROUTES\]|\[SESSION-MANAGER\]|\[SDK-AGENT\]'
|
||||
```
|
||||
|
||||
### 2.3 Check Database State
|
||||
|
||||
**Query 1 - Check sessions table**:
|
||||
```bash
|
||||
cd ~/.claude-mem
|
||||
sqlite3 claude-mem.db "SELECT id, claude_session_id, sdk_session_id, status, started_at FROM sdk_sessions ORDER BY id DESC LIMIT 10;"
|
||||
```
|
||||
|
||||
**Expected**: Same `claude_session_id` for all 3 prompts
|
||||
|
||||
**Query 2 - Check user prompts table**:
|
||||
```bash
|
||||
sqlite3 claude-mem.db "SELECT claude_session_id, prompt_number, created_at FROM user_prompts ORDER BY created_at DESC LIMIT 10;"
|
||||
```
|
||||
|
||||
**Expected**: Same `claude_session_id` with prompt_number: 1, 2, 3
|
||||
|
||||
### 2.4 Analyze Data Flow
|
||||
|
||||
For each prompt (1, 2, 3), trace in logs:
|
||||
|
||||
1. **NEW-HOOK** receives `session_id` from Claude Code
|
||||
2. **SESSION-ROUTES** receives `claudeSessionId` in API call
|
||||
3. **SESSION-ROUTES** creates/gets `sessionDbId`
|
||||
4. **SESSION-ROUTES** calculates `promptNumber`
|
||||
5. **SESSION-MANAGER** fetches/creates session with `claudeSessionId`
|
||||
6. **SDK-AGENT** uses `claudeSessionId` as resume parameter
|
||||
7. **SDK-AGENT** selects INIT vs CONTINUATION prompt
|
||||
|
||||
**Key Questions to Answer**:
|
||||
- [ ] Does `session_id` from hook stay the same across all 3 prompts?
|
||||
- [ ] Does `claudeSessionId` match across all log entries for same conversation?
|
||||
- [ ] Does `promptNumber` increment: 1, 2, 3?
|
||||
- [ ] Does `lastPromptNumber` match `promptNumber` in SessionManager?
|
||||
- [ ] Does SDK-AGENT receive correct `resume` parameter on prompts 2+?
|
||||
- [ ] Does SDK-AGENT select CONTINUATION prompt for prompts 2+?
|
||||
|
||||
**Success Criteria**:
|
||||
- [ ] Logs collected for 3 test prompts
|
||||
- [ ] Database queries run and results saved
|
||||
- [ ] Data flow analysis completed
|
||||
- [ ] Failure point identified
|
||||
|
||||
**Handoff to Phase 3**: Document exact failure point (which log entry shows incorrect value) and move to fix implementation
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Implement Fix Based on Findings
|
||||
|
||||
**Goal**: Fix the identified root cause of session continuity failure.
|
||||
|
||||
**Prerequisites**: Phase 2 completed, failure point identified from logs/database
|
||||
|
||||
**Common Fix Scenarios**:
|
||||
|
||||
### Scenario A: Hook Receives Different `session_id` Each Time
|
||||
|
||||
**Symptom in Logs**:
|
||||
```
|
||||
[NEW-HOOK] Received hook input: { session_id: 'abc-123', ... } // Prompt 1
|
||||
[NEW-HOOK] Received hook input: { session_id: 'def-456', ... } // Prompt 2 - DIFFERENT!
|
||||
```
|
||||
|
||||
**Root Cause**: Hook not receiving consistent session ID from Claude Code
|
||||
|
||||
**Fix Location**: This is external to codebase - investigate Claude Code hook configuration or report bug
|
||||
|
||||
**Action**: Create GitHub issue in claude-code repo with evidence
|
||||
|
||||
### Scenario B: `promptNumber` Not Passed or Calculated Correctly
|
||||
|
||||
**Symptom in Logs**:
|
||||
```
|
||||
[SESSION-ROUTES] Calculated promptNumber: { promptNumber: 1, currentCount: 1 } // Prompt 2 - WRONG!
|
||||
```
|
||||
|
||||
**Root Cause**: User prompt not being saved to database, or count query failing
|
||||
|
||||
**Fix Location**: `src/services/worker/http/routes/SessionRoutes.ts` line 520
|
||||
|
||||
**Fix**:
|
||||
```typescript
|
||||
// Add error handling around saveUserPrompt
|
||||
try {
|
||||
this.dbManager.getSessionStore().saveUserPrompt(
|
||||
claudeSessionId,
|
||||
promptNumber,
|
||||
cleanedPrompt
|
||||
);
|
||||
console.log('[SESSION-ROUTES] Successfully saved user prompt:', {
|
||||
claudeSessionId,
|
||||
promptNumber
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[SESSION-ROUTES] Failed to save user prompt:', error);
|
||||
throw new Error(`Failed to save user prompt: ${error.message}`);
|
||||
}
|
||||
```
|
||||
|
||||
### Scenario C: Session Manager Uses Wrong Fallback Logic
|
||||
|
||||
**Symptom in Logs**:
|
||||
```
|
||||
[SESSION-MANAGER] Creating new session object: { lastPromptNumber: 1 } // Prompt 2 - WRONG!
|
||||
```
|
||||
|
||||
**Root Cause**: Fragile `||` operator causing incorrect fallback when `promptNumber` is valid
|
||||
|
||||
**Fix Location**: `src/services/worker/SessionManager.ts` line 116
|
||||
|
||||
**Fix**:
|
||||
```typescript
|
||||
// Replace fragile || with explicit undefined check
|
||||
lastPromptNumber: promptNumber !== undefined
|
||||
? promptNumber
|
||||
: this.dbManager.getSessionStore().getPromptNumberFromUserPrompts(dbSession.claude_session_id),
|
||||
```
|
||||
|
||||
### Scenario D: Database Session Not Found
|
||||
|
||||
**Symptom in Logs**:
|
||||
```
|
||||
[SESSION-MANAGER] Fetched session from database: { claude_session_id: undefined }
|
||||
```
|
||||
|
||||
**Root Cause**: `createSDKSession` INSERT failed silently, or session was deleted
|
||||
|
||||
**Fix Location**: `src/services/sqlite/SessionStore.ts` line 1086-1101
|
||||
|
||||
**Fix**:
|
||||
```typescript
|
||||
// Add validation after INSERT OR IGNORE
|
||||
const result = this.db.prepare(`
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(claudeSessionId, claudeSessionId, project, userPrompt, now, nowEpoch, 'active');
|
||||
|
||||
// Verify session exists
|
||||
const row = this.db.prepare('SELECT id FROM sdk_sessions WHERE claude_session_id = ?')
|
||||
.get(claudeSessionId);
|
||||
|
||||
if (!row) {
|
||||
throw new Error(`Failed to create or retrieve SDK session for claudeSessionId: ${claudeSessionId}`);
|
||||
}
|
||||
|
||||
return row.id;
|
||||
```
|
||||
|
||||
### Scenario E: SDK Agent Receives Empty `claudeSessionId`
|
||||
|
||||
**Symptom in Logs**:
|
||||
```
|
||||
[SDK-AGENT] Starting SDK query with: { claudeSessionId: undefined, resume_parameter: undefined }
|
||||
```
|
||||
|
||||
**Root Cause**: SessionManager created session object with missing `claudeSessionId`
|
||||
|
||||
**Fix Location**: `src/services/worker/SessionManager.ts` line 109
|
||||
|
||||
**Fix**:
|
||||
```typescript
|
||||
// Add validation before using database values
|
||||
if (!dbSession.claude_session_id) {
|
||||
throw new Error(`Database session ${sessionDbId} has no claude_session_id`);
|
||||
}
|
||||
|
||||
session = {
|
||||
sessionDbId,
|
||||
claudeSessionId: dbSession.claude_session_id,
|
||||
// ... rest of session object
|
||||
};
|
||||
```
|
||||
|
||||
**Success Criteria**:
|
||||
- [ ] Fix implemented at identified failure point
|
||||
- [ ] Validation added to fail loudly on errors
|
||||
- [ ] Build succeeds
|
||||
- [ ] Worker service restarts successfully
|
||||
|
||||
**Handoff to Phase 4**: Build and deploy fix, then run verification tests
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Verify Fix and Test Session Continuity
|
||||
|
||||
**Goal**: Confirm session continuity is working correctly after fix.
|
||||
|
||||
**Prerequisites**: Phase 3 completed, fix deployed, worker service running
|
||||
|
||||
**Verification Procedure**:
|
||||
|
||||
### 4.1 Run Full Test Conversation
|
||||
|
||||
In a fresh Claude Code session:
|
||||
|
||||
1. **Prompt 1**: "This is test prompt one for session continuity"
|
||||
2. **Prompt 2**: "This is test prompt two, continuing the session"
|
||||
3. **Prompt 3**: "This is test prompt three, still continuing"
|
||||
4. **Prompt 4**: "Final test prompt four"
|
||||
|
||||
### 4.2 Check Logs
|
||||
|
||||
Verify in worker logs:
|
||||
|
||||
**All prompts show same `session_id`**:
|
||||
```
|
||||
[NEW-HOOK] Received hook input: { session_id: 'abc-123' } // All 4 prompts
|
||||
```
|
||||
|
||||
**Prompt numbers increment**:
|
||||
```
|
||||
[SESSION-ROUTES] Calculated promptNumber: { promptNumber: 1 } // Prompt 1
|
||||
[SESSION-ROUTES] Calculated promptNumber: { promptNumber: 2 } // Prompt 2
|
||||
[SESSION-ROUTES] Calculated promptNumber: { promptNumber: 3 } // Prompt 3
|
||||
[SESSION-ROUTES] Calculated promptNumber: { promptNumber: 4 } // Prompt 4
|
||||
```
|
||||
|
||||
**SDK Agent uses continuation prompts**:
|
||||
```
|
||||
[SDK-AGENT] Creating message generator: { promptType: 'INIT' } // Prompt 1
|
||||
[SDK-AGENT] Creating message generator: { promptType: 'CONTINUATION' } // Prompt 2
|
||||
[SDK-AGENT] Creating message generator: { promptType: 'CONTINUATION' } // Prompt 3
|
||||
[SDK-AGENT] Creating message generator: { promptType: 'CONTINUATION' } // Prompt 4
|
||||
```
|
||||
|
||||
### 4.3 Verify Database State
|
||||
|
||||
**Check sessions table**:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT id, claude_session_id, sdk_session_id FROM sdk_sessions ORDER BY id DESC LIMIT 5;"
|
||||
```
|
||||
|
||||
**Expected**: Only ONE session record for the 4 prompts, `claude_session_id` and `sdk_session_id` are identical
|
||||
|
||||
**Check user_prompts table**:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT claude_session_id, prompt_number, created_at FROM user_prompts ORDER BY created_at DESC LIMIT 5;"
|
||||
```
|
||||
|
||||
**Expected**: 4 records with same `claude_session_id`, prompt_number values: 4, 3, 2, 1
|
||||
|
||||
### 4.4 Functional Test
|
||||
|
||||
Verify actual session continuity behavior:
|
||||
|
||||
1. **Prompt 1**: "My favorite color is blue"
|
||||
2. **Prompt 2**: "What is my favorite color?"
|
||||
- **Expected**: Response mentions "blue"
|
||||
3. **Prompt 3**: "Change it to red"
|
||||
4. **Prompt 4**: "What is my favorite color now?"
|
||||
- **Expected**: Response mentions "red"
|
||||
|
||||
**Success Criteria**:
|
||||
- [x] Same `session_id` across all 4 prompts in logs
|
||||
- [x] Prompt numbers increment: 1, 2, 3, 4
|
||||
- [x] INIT prompt only for first prompt
|
||||
- [x] CONTINUATION prompts for prompts 2, 3, 4
|
||||
- [x] Only one session record in database
|
||||
- [x] Four user_prompts records with incremental prompt_number
|
||||
- [x] Functional test shows session continuity working
|
||||
|
||||
**Handoff to Phase 5**: If all criteria pass, proceed to cleanup. If any fail, return to Phase 2 with new diagnostic focus.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Cleanup and Documentation
|
||||
|
||||
**Goal**: Remove excessive logging, update documentation, close issues.
|
||||
|
||||
**Prerequisites**: Phase 4 completed successfully, session continuity verified working
|
||||
|
||||
**Cleanup Steps**:
|
||||
|
||||
### 5.1 Reduce Logging Verbosity (Optional)
|
||||
|
||||
You can either:
|
||||
- **Keep all diagnostic logging** for future debugging (recommended)
|
||||
- **Remove logging** to reduce noise in production logs
|
||||
- **Convert to debug level** if logging framework supports it
|
||||
|
||||
If removing logging, remove the `console.log` statements added in Phase 1 from:
|
||||
- `src/hooks/new-hook.ts`
|
||||
- `src/services/worker/http/routes/SessionRoutes.ts`
|
||||
- `src/services/worker/SessionManager.ts`
|
||||
- `src/services/worker/SDKAgent.ts`
|
||||
|
||||
### 5.2 Update Documentation
|
||||
|
||||
If the fix revealed any architectural insights, update:
|
||||
- `CLAUDE.md` - Add any new gotchas or patterns discovered
|
||||
- `README.md` - Update if user-facing behavior changed
|
||||
- Code comments - Document the fix rationale
|
||||
|
||||
### 5.3 Create Regression Test (Future Work)
|
||||
|
||||
Consider adding automated test:
|
||||
```typescript
|
||||
describe('Session Continuity', () => {
|
||||
it('should use same session ID across multiple prompts', async () => {
|
||||
// Test that verifies session ID propagation
|
||||
});
|
||||
|
||||
it('should increment prompt numbers correctly', async () => {
|
||||
// Test that verifies prompt number calculation
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### 5.4 Close Related Issues
|
||||
|
||||
Search GitHub for related issues:
|
||||
```bash
|
||||
gh issue list --search "session continuity" --state open
|
||||
gh issue list --search "session persistence" --state open
|
||||
gh issue list --search "new session" --state open
|
||||
```
|
||||
|
||||
Close with comment explaining the fix.
|
||||
|
||||
**Success Criteria**:
|
||||
- [ ] Logging cleaned up as desired
|
||||
- [ ] Documentation updated
|
||||
- [ ] Related GitHub issues closed
|
||||
- [ ] No regressions introduced
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Key Files and What They Do
|
||||
|
||||
| File | Purpose | Critical Lines |
|
||||
|------|---------|----------------|
|
||||
| `src/hooks/new-hook.ts` | Hook entry point, receives session_id from Claude Code | 24, 34, 46-47, 63-68 |
|
||||
| `src/services/worker/http/routes/SessionRoutes.ts` | HTTP endpoints for session init, calculates prompt numbers | 482-533, 171-227 |
|
||||
| `src/services/sqlite/SessionStore.ts` | Database operations for sessions and user prompts | 1086-1101, 1053-1058 |
|
||||
| `src/services/worker/SessionManager.ts` | In-memory session management, bridges DB and SDK | 49-141, esp. 109, 116 |
|
||||
| `src/services/worker/SDKAgent.ts` | SDK integration, sends resume parameter and prompts | 68-77, 195-218, 200-202 |
|
||||
| `src/sdk/prompts.ts` | Init and continuation prompt templates | 30-87, 169-229 |
|
||||
|
||||
### Build and Deploy Commands
|
||||
|
||||
```bash
|
||||
# Build TypeScript
|
||||
npm run build
|
||||
|
||||
# Sync to marketplace and restart worker
|
||||
npm run build-and-sync
|
||||
|
||||
# Restart worker only
|
||||
killall bun
|
||||
bun ~/.claude/plugins/marketplaces/thedotmack/scripts/worker-service.cjs &
|
||||
|
||||
# Check worker is running
|
||||
curl http://localhost:37777/health
|
||||
```
|
||||
|
||||
### Database Queries
|
||||
|
||||
```bash
|
||||
# Check sessions
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT * FROM sdk_sessions ORDER BY id DESC LIMIT 10;"
|
||||
|
||||
# Check user prompts
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT * FROM user_prompts ORDER BY created_at DESC LIMIT 10;"
|
||||
|
||||
# Count prompts per session
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT claude_session_id, COUNT(*) as prompt_count FROM user_prompts GROUP BY claude_session_id ORDER BY prompt_count DESC LIMIT 10;"
|
||||
```
|
||||
|
||||
### Debugging Tips
|
||||
|
||||
1. **Check worker is running**: `curl http://localhost:37777/health`
|
||||
2. **View worker logs**: `tail -f /tmp/worker-logs.txt`
|
||||
3. **Check hook output**: Logs appear in Claude Code's stderr
|
||||
4. **Database locked**: `killall bun` then restart worker
|
||||
5. **Stale build**: `rm -rf plugin/scripts/*.js && npm run build`
|
||||
|
||||
---
|
||||
|
||||
## Phase Execution Checklist
|
||||
|
||||
Use this checklist when executing phases in new chat contexts:
|
||||
|
||||
**Phase 1: Diagnostic Logging**
|
||||
- [ ] Read this plan document
|
||||
- [ ] Read the 4 files to modify
|
||||
- [ ] Add all 15+ log points
|
||||
- [ ] Build with `npm run build-and-sync`
|
||||
- [ ] Verify worker restarts
|
||||
- [ ] Mark phase complete, handoff to Phase 2
|
||||
|
||||
**Phase 2: Test and Gather Data**
|
||||
- [ ] Read Phase 2 section
|
||||
- [ ] Run 3 test prompts
|
||||
- [ ] Collect and save logs
|
||||
- [ ] Run database queries
|
||||
- [ ] Trace data flow
|
||||
- [ ] Identify failure point
|
||||
- [ ] Document failure point
|
||||
- [ ] Mark phase complete, handoff to Phase 3
|
||||
|
||||
**Phase 3: Implement Fix**
|
||||
- [ ] Read Phase 3 section
|
||||
- [ ] Review failure point from Phase 2
|
||||
- [ ] Select applicable scenario
|
||||
- [ ] Implement fix
|
||||
- [ ] Add validation
|
||||
- [ ] Build and deploy
|
||||
- [ ] Mark phase complete, handoff to Phase 4
|
||||
|
||||
**Phase 4: Verify Fix**
|
||||
- [ ] Read Phase 4 section
|
||||
- [ ] Run 4 test prompts
|
||||
- [ ] Check logs for correct behavior
|
||||
- [ ] Verify database state
|
||||
- [ ] Run functional test
|
||||
- [ ] All success criteria pass
|
||||
- [ ] Mark phase complete, handoff to Phase 5
|
||||
|
||||
**Phase 5: Cleanup**
|
||||
- [ ] Read Phase 5 section
|
||||
- [ ] Clean up logging (optional)
|
||||
- [ ] Update documentation
|
||||
- [ ] Close GitHub issues
|
||||
- [ ] Mark phase complete
|
||||
- [ ] Session continuity regression FIX COMPLETE ✅
|
||||
|
||||
---
|
||||
|
||||
## Context for New Chat Sessions
|
||||
|
||||
When starting a new phase, provide this context:
|
||||
|
||||
**I'm working on Phase [X] of the Session Continuity Regression Fix for claude-mem.**
|
||||
|
||||
**Background**: Session continuity is broken - each prompt creates a new session instead of continuing. This has been a recurring issue for 3 months. The root cause is that session SDK ID is not being propagated correctly from new-hook through to SDKAgent.
|
||||
|
||||
**Current Status**: [Briefly describe what previous phases accomplished]
|
||||
|
||||
**This Phase Goal**: [Copy the goal from the phase section]
|
||||
|
||||
**Plan Document**: Read `/Users/alexnewman/Scripts/claude-mem/PLAN-SESSION-CONTINUITY-FIX.md` for full context.
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
**Overall Fix Success**:
|
||||
- [ ] Same session ID used across multiple prompts in one conversation
|
||||
- [ ] Prompt numbers increment correctly (1, 2, 3, ...)
|
||||
- [ ] Init prompt only sent on first prompt
|
||||
- [ ] Continuation prompts sent on subsequent prompts
|
||||
- [ ] SDK receives correct resume parameter
|
||||
- [ ] Only one session record created per conversation
|
||||
- [ ] Functional session continuity test passes
|
||||
- [ ] No new regressions introduced
|
||||
|
||||
**Regression Prevention**:
|
||||
- [ ] Validation added to fail loudly on errors
|
||||
- [ ] No silent fallbacks that hide bugs
|
||||
- [ ] Database queries verified
|
||||
- [ ] Session ID propagation explicitly tested
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-12-27
|
||||
**Author**: Claude (investigating 3-month recurring session continuity regression)
|
||||
@@ -106,7 +106,7 @@ pm2 logs claude-mem-worker # View logs
|
||||
```bash
|
||||
npm run worker:start # Start worker
|
||||
npm run worker:stop # Stop worker
|
||||
claude-mem restart # Restart worker
|
||||
npm run worker:restart # Restart worker
|
||||
npm run worker:status # Check status
|
||||
npm run worker:logs # View logs
|
||||
```
|
||||
@@ -305,7 +305,7 @@ No migration logic runs on subsequent sessions.
|
||||
| `pm2 list` | `npm run worker:status` | Shows worker status |
|
||||
| `pm2 start <script>` | `npm run worker:start` | Start worker |
|
||||
| `pm2 stop claude-mem-worker` | `npm run worker:stop` | Stop worker |
|
||||
| `pm2 restart claude-mem-worker` | `claude-mem restart` | Restart worker |
|
||||
| `pm2 restart claude-mem-worker` | `npm run worker:restart` | Restart worker |
|
||||
| `pm2 delete claude-mem-worker` | `npm run worker:stop` | Remove worker |
|
||||
| `pm2 logs claude-mem-worker` | `npm run worker:logs` | View logs |
|
||||
| `pm2 describe claude-mem-worker` | `npm run worker:status` | Detailed status |
|
||||
@@ -451,7 +451,7 @@ pm2 save # Persist the deletion
|
||||
rm ~/.claude-mem/.pm2-migrated
|
||||
|
||||
# Restart worker
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Scenario 2: Stale PID File (Process Dead)
|
||||
@@ -483,7 +483,7 @@ lsof -i :37777
|
||||
kill -9 <PID>
|
||||
|
||||
# Restart worker
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Common Error Messages
|
||||
|
||||
@@ -416,7 +416,7 @@ If searches fail, check worker service:
|
||||
|
||||
```bash
|
||||
npm run worker:status # Check status
|
||||
claude-mem restart # Restart worker
|
||||
npm run worker:restart # Restart worker
|
||||
npm run worker:logs # View logs
|
||||
```
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ The worker service is a long-running HTTP API built with Express.js and managed
|
||||
|
||||
## REST API Endpoints
|
||||
|
||||
The worker service exposes 20 HTTP endpoints organized into five categories:
|
||||
The worker service exposes 22 HTTP endpoints organized into six categories:
|
||||
|
||||
### Viewer & Health Endpoints
|
||||
|
||||
@@ -385,9 +385,106 @@ POST /api/settings
|
||||
}
|
||||
```
|
||||
|
||||
### Queue Management Endpoints
|
||||
|
||||
#### 16. Get Pending Queue Status
|
||||
```
|
||||
GET /api/pending-queue
|
||||
```
|
||||
|
||||
**Purpose**: View current processing queue status and identify stuck messages
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"queue": {
|
||||
"messages": [
|
||||
{
|
||||
"id": 123,
|
||||
"session_db_id": 45,
|
||||
"claude_session_id": "abc123",
|
||||
"message_type": "observation",
|
||||
"status": "pending",
|
||||
"retry_count": 0,
|
||||
"created_at_epoch": 1730886600000,
|
||||
"started_processing_at_epoch": null,
|
||||
"completed_at_epoch": null
|
||||
}
|
||||
],
|
||||
"totalPending": 5,
|
||||
"totalProcessing": 2,
|
||||
"totalFailed": 0,
|
||||
"stuckCount": 1
|
||||
},
|
||||
"recentlyProcessed": [
|
||||
{
|
||||
"id": 122,
|
||||
"session_db_id": 44,
|
||||
"status": "processed",
|
||||
"completed_at_epoch": 1730886500000
|
||||
}
|
||||
],
|
||||
"sessionsWithPendingWork": [44, 45, 46]
|
||||
}
|
||||
```
|
||||
|
||||
**Status Definitions**:
|
||||
- `pending`: Message queued, not yet processed
|
||||
- `processing`: Message currently being processed by SDK agent
|
||||
- `processed`: Message completed successfully
|
||||
- `failed`: Message failed after max retry attempts (3 by default)
|
||||
|
||||
**Stuck Detection**: Messages in `processing` status for >5 minutes are considered stuck and included in `stuckCount`
|
||||
|
||||
**Use Case**: Check queue health after worker crashes or restarts to identify unprocessed observations
|
||||
|
||||
#### 17. Trigger Manual Recovery
|
||||
```
|
||||
POST /api/pending-queue/process
|
||||
```
|
||||
|
||||
**Purpose**: Manually trigger processing of pending queues (replaces automatic recovery in v5.x+)
|
||||
|
||||
**Request Body**:
|
||||
```json
|
||||
{
|
||||
"sessionLimit": 10
|
||||
}
|
||||
```
|
||||
|
||||
**Body Parameters**:
|
||||
- `sessionLimit` (optional): Maximum number of sessions to process (default: 10, max: 100)
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"totalPendingSessions": 15,
|
||||
"sessionsStarted": 10,
|
||||
"sessionsSkipped": 2,
|
||||
"startedSessionIds": [44, 45, 46, 47, 48, 49, 50, 51, 52, 53]
|
||||
}
|
||||
```
|
||||
|
||||
**Response Fields**:
|
||||
- `totalPendingSessions`: Total sessions with pending messages in database
|
||||
- `sessionsStarted`: Number of sessions we started processing this request
|
||||
- `sessionsSkipped`: Sessions already actively processing (not restarted)
|
||||
- `startedSessionIds`: Database IDs of sessions started
|
||||
|
||||
**Behavior**:
|
||||
- Processes up to `sessionLimit` sessions with pending work
|
||||
- Skips sessions already actively processing (prevents duplicate agents)
|
||||
- Starts non-blocking SDK agents for each session
|
||||
- Returns immediately with status (processing continues in background)
|
||||
|
||||
**Use Case**: Manually recover stuck observations after worker crashes, or when automatic recovery was disabled
|
||||
|
||||
**Recovery Strategy Note**: As of v5.x, automatic recovery on worker startup is disabled by default. Users must manually trigger recovery using this endpoint or the CLI tool (`bun scripts/check-pending-queue.ts`) to maintain explicit control over reprocessing.
|
||||
|
||||
### Session Management Endpoints
|
||||
|
||||
#### 16. Initialize Session
|
||||
#### 19. Initialize Session
|
||||
```
|
||||
POST /sessions/:sessionDbId/init
|
||||
```
|
||||
@@ -408,7 +505,7 @@ POST /sessions/:sessionDbId/init
|
||||
}
|
||||
```
|
||||
|
||||
#### 17. Add Observation
|
||||
#### 20. Add Observation
|
||||
```
|
||||
POST /sessions/:sessionDbId/observations
|
||||
```
|
||||
@@ -431,7 +528,7 @@ POST /sessions/:sessionDbId/observations
|
||||
}
|
||||
```
|
||||
|
||||
#### 18. Generate Summary
|
||||
#### 21. Generate Summary
|
||||
```
|
||||
POST /sessions/:sessionDbId/summarize
|
||||
```
|
||||
@@ -451,7 +548,7 @@ POST /sessions/:sessionDbId/summarize
|
||||
}
|
||||
```
|
||||
|
||||
#### 19. Session Status
|
||||
#### 22. Session Status
|
||||
```
|
||||
GET /sessions/:sessionDbId/status
|
||||
```
|
||||
@@ -466,7 +563,7 @@ GET /sessions/:sessionDbId/status
|
||||
}
|
||||
```
|
||||
|
||||
#### 20. Delete Session
|
||||
#### 23. Delete Session
|
||||
```
|
||||
DELETE /sessions/:sessionDbId
|
||||
```
|
||||
@@ -500,7 +597,7 @@ npm run worker:start
|
||||
npm run worker:stop
|
||||
|
||||
# Restart worker
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
|
||||
# View logs
|
||||
npm run worker:logs
|
||||
|
||||
@@ -13,12 +13,35 @@ Settings are managed in `~/.claude-mem/settings.json`. The file is auto-created
|
||||
|
||||
| Setting | Default | Description |
|
||||
|-------------------------------|---------------------------------|---------------------------------------|
|
||||
| `CLAUDE_MEM_MODEL` | `sonnet` | AI model for processing observations |
|
||||
| `CLAUDE_MEM_MODEL` | `sonnet` | AI model for processing observations (when using Claude) |
|
||||
| `CLAUDE_MEM_PROVIDER` | `claude` | AI provider: `claude`, `gemini`, or `openrouter` |
|
||||
| `CLAUDE_MEM_MODE` | `code` | Active mode profile (e.g., `code--es`, `email-investigation`) |
|
||||
| `CLAUDE_MEM_CONTEXT_OBSERVATIONS` | `50` | Number of observations to inject |
|
||||
| `CLAUDE_MEM_WORKER_PORT` | `37777` | Worker service port |
|
||||
| `CLAUDE_MEM_SKIP_TOOLS` | `ListMcpResourcesTool,SlashCommand,Skill,TodoWrite,AskUserQuestion` | Comma-separated tools to exclude from observations |
|
||||
|
||||
### Gemini Provider Settings
|
||||
|
||||
| Setting | Default | Description |
|
||||
|-------------------------------|---------------------------------|---------------------------------------|
|
||||
| `CLAUDE_MEM_GEMINI_API_KEY` | — | Gemini API key ([get free key](https://aistudio.google.com/app/apikey)) |
|
||||
| `CLAUDE_MEM_GEMINI_MODEL` | `gemini-2.5-flash-lite` | Gemini model: `gemini-2.5-flash-lite`, `gemini-2.5-flash`, `gemini-3-flash` |
|
||||
|
||||
See [Gemini Provider](usage/gemini-provider) for detailed configuration and free tier information.
|
||||
|
||||
### OpenRouter Provider Settings
|
||||
|
||||
| Setting | Default | Description |
|
||||
|----------------------------------------------|-----------------------------|---------------------------------------|
|
||||
| `CLAUDE_MEM_OPENROUTER_API_KEY` | — | OpenRouter API key ([get key](https://openrouter.ai/keys)) |
|
||||
| `CLAUDE_MEM_OPENROUTER_MODEL` | `xiaomi/mimo-v2-flash:free` | Model identifier (supports 100+ models) |
|
||||
| `CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES` | `20` | Max messages in conversation history |
|
||||
| `CLAUDE_MEM_OPENROUTER_MAX_TOKENS` | `100000` | Token budget safety limit |
|
||||
| `CLAUDE_MEM_OPENROUTER_SITE_URL` | — | Optional: URL for analytics |
|
||||
| `CLAUDE_MEM_OPENROUTER_APP_NAME` | `claude-mem` | Optional: App name for analytics |
|
||||
|
||||
See [OpenRouter Provider](usage/openrouter-provider) for detailed configuration, free model list, and usage guide.
|
||||
|
||||
### System Configuration
|
||||
|
||||
| Setting | Default | Description |
|
||||
@@ -117,7 +140,6 @@ ${CLAUDE_PLUGIN_ROOT}/
|
||||
│ ├── new-hook.js # Session creation hook
|
||||
│ ├── save-hook.js # Observation capture hook
|
||||
│ ├── summary-hook.js # Summary generation hook
|
||||
│ ├── cleanup-hook.js # Session cleanup hook
|
||||
│ ├── worker-service.cjs # Worker service (CJS)
|
||||
│ └── mcp-server.cjs # MCP search server (CJS)
|
||||
└── ui/
|
||||
@@ -162,13 +184,6 @@ Hooks are configured in `plugin/hooks/hooks.json`:
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}],
|
||||
"SessionEnd": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/cleanup-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
@@ -343,7 +358,7 @@ Edit `~/.claude-mem/settings.json`:
|
||||
|
||||
Then restart the worker:
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Custom Model
|
||||
@@ -358,7 +373,7 @@ Edit `~/.claude-mem/settings.json`:
|
||||
Then restart the worker:
|
||||
```bash
|
||||
export CLAUDE_MEM_MODEL=opus
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Custom Skip Tools
|
||||
@@ -415,7 +430,7 @@ Enable debug logging:
|
||||
|
||||
```bash
|
||||
export DEBUG=claude-mem:*
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
@@ -433,7 +448,7 @@ npm run worker:logs
|
||||
|
||||
1. Restart worker after changes:
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
2. Verify environment variables:
|
||||
@@ -467,7 +482,7 @@ If port 37777 is already in use:
|
||||
|
||||
2. Restart worker:
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
3. Verify new port:
|
||||
|
||||
+133
-29
@@ -165,7 +165,7 @@ npm run build
|
||||
1. Make changes to React components in `src/ui/viewer/`
|
||||
2. Build: `npm run build`
|
||||
3. Sync to installed plugin: `npm run sync-marketplace`
|
||||
4. Restart worker: `claude-mem restart`
|
||||
4. Restart worker: `npm run worker:restart`
|
||||
5. Refresh browser at http://localhost:37777
|
||||
|
||||
**Hot Reload**: Not currently supported. Full rebuild + restart required for changes.
|
||||
@@ -371,45 +371,149 @@ npm test
|
||||
|
||||
## Testing
|
||||
|
||||
### Running Tests
|
||||
### Testing Philosophy
|
||||
|
||||
Claude-mem relies on **real-world usage and manual testing** rather than traditional unit tests. The project philosophy prioritizes:
|
||||
|
||||
1. **Manual verification** - Testing features in actual Claude Code sessions
|
||||
2. **Integration testing** - Running the full system end-to-end
|
||||
3. **Database inspection** - Verifying data correctness via SQLite queries
|
||||
4. **CLI tools** - Interactive tools for checking system state
|
||||
5. **Observability** - Comprehensive logging and worker health checks
|
||||
|
||||
This approach was chosen because:
|
||||
- Hook behavior depends heavily on Claude Code's runtime environment
|
||||
- SDK interactions require real API calls and responses
|
||||
- SQLite and Bun runtime provide stability guarantees
|
||||
- Manual testing catches integration issues that unit tests miss
|
||||
|
||||
### Manual Testing Workflow
|
||||
|
||||
When developing new features:
|
||||
|
||||
1. **Build and sync**:
|
||||
```bash
|
||||
npm run build
|
||||
npm run sync-marketplace
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
2. **Test in real session**:
|
||||
- Start Claude Code
|
||||
- Trigger the feature you're testing
|
||||
- Verify expected behavior
|
||||
|
||||
3. **Check database state**:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT * FROM your_table;"
|
||||
```
|
||||
|
||||
4. **Monitor worker logs**:
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
5. **Verify queue health** (for recovery features):
|
||||
```bash
|
||||
bun scripts/check-pending-queue.ts
|
||||
```
|
||||
|
||||
### Testing Tools
|
||||
|
||||
**Health Checks**:
|
||||
```bash
|
||||
# All tests
|
||||
npm test
|
||||
# Worker status
|
||||
npm run worker:status
|
||||
|
||||
# Specific test file
|
||||
node --test tests/your-test.test.ts
|
||||
# Queue inspection
|
||||
curl http://localhost:37777/api/pending-queue
|
||||
|
||||
# With coverage (if configured)
|
||||
npm test -- --coverage
|
||||
# Database integrity
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
|
||||
### Writing Tests
|
||||
**Hook Testing**:
|
||||
```bash
|
||||
# Test context hook manually
|
||||
echo '{"session_id":"test-123","cwd":"'$(pwd)'","source":"startup"}' | node plugin/scripts/context-hook.js
|
||||
|
||||
Create test files in `tests/`:
|
||||
|
||||
```typescript
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert';
|
||||
|
||||
describe('YourFeature', () => {
|
||||
it('should do something', () => {
|
||||
// Test implementation
|
||||
assert.strictEqual(result, expected);
|
||||
});
|
||||
});
|
||||
# Test new hook
|
||||
echo '{"session_id":"test-123","cwd":"'$(pwd)'","prompt":"test"}' | node plugin/scripts/new-hook.js
|
||||
```
|
||||
|
||||
### Test Database
|
||||
**Data Verification**:
|
||||
```bash
|
||||
# Check recent observations
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT id, tool_name, created_at
|
||||
FROM observations
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT 10;
|
||||
"
|
||||
|
||||
Use a separate test database:
|
||||
|
||||
```typescript
|
||||
import { SessionStore } from '../src/services/sqlite/SessionStore';
|
||||
|
||||
const store = new SessionStore(':memory:'); // In-memory database
|
||||
# Check summaries
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT id, request, completed
|
||||
FROM session_summaries
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT 5;
|
||||
"
|
||||
```
|
||||
|
||||
### Recovery Feature Testing
|
||||
|
||||
For manual recovery features specifically:
|
||||
|
||||
1. **Simulate stuck messages**:
|
||||
```bash
|
||||
# Manually create stuck message (for testing only)
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
UPDATE pending_messages
|
||||
SET status = 'processing',
|
||||
started_processing_at_epoch = strftime('%s', 'now', '-10 minutes') * 1000
|
||||
WHERE id = 123;
|
||||
"
|
||||
```
|
||||
|
||||
2. **Test recovery**:
|
||||
```bash
|
||||
bun scripts/check-pending-queue.ts
|
||||
```
|
||||
|
||||
3. **Verify results**:
|
||||
```bash
|
||||
curl http://localhost:37777/api/pending-queue | jq '.queue'
|
||||
```
|
||||
|
||||
### Regression Testing
|
||||
|
||||
Before releasing:
|
||||
|
||||
1. **Test all hook triggers**:
|
||||
- SessionStart: Start new Claude Code session
|
||||
- UserPromptSubmit: Submit a prompt
|
||||
- PostToolUse: Use a tool like Read
|
||||
- Summary: Let session complete
|
||||
- SessionEnd: Close Claude Code
|
||||
|
||||
2. **Test core features**:
|
||||
- Context injection (recent sessions appear)
|
||||
- Observation processing (summaries generated)
|
||||
- MCP search tools (search returns results)
|
||||
- Viewer UI (loads at http://localhost:37777)
|
||||
- Manual recovery (stuck messages recovered)
|
||||
|
||||
3. **Test edge cases**:
|
||||
- Worker crash recovery
|
||||
- Database locks
|
||||
- Port conflicts
|
||||
- Large databases
|
||||
|
||||
4. **Cross-platform** (if applicable):
|
||||
- macOS
|
||||
- Linux
|
||||
- Windows
|
||||
|
||||
## Code Style
|
||||
|
||||
### TypeScript Guidelines
|
||||
@@ -456,7 +560,7 @@ export async function createObservation(
|
||||
|
||||
```bash
|
||||
export DEBUG=claude-mem:*
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
|
||||
@@ -36,10 +36,13 @@
|
||||
"introduction",
|
||||
"installation",
|
||||
"usage/getting-started",
|
||||
"usage/openrouter-provider",
|
||||
"usage/gemini-provider",
|
||||
"usage/search-tools",
|
||||
"usage/claude-desktop",
|
||||
"usage/private-tags",
|
||||
"usage/export-import",
|
||||
"usage/manual-recovery",
|
||||
"beta-features",
|
||||
"endless-mode"
|
||||
]
|
||||
|
||||
@@ -94,7 +94,7 @@ git checkout beta/endless-mode
|
||||
npm install
|
||||
|
||||
# Restart the worker
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
**To return to stable:**
|
||||
@@ -103,7 +103,7 @@ claude-mem restart
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
git checkout main
|
||||
npm install
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
@@ -534,7 +534,7 @@ npm run worker:status
|
||||
npm run worker:logs
|
||||
|
||||
# Restart
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
|
||||
# Stop
|
||||
npm run worker:stop
|
||||
|
||||
@@ -57,7 +57,7 @@ CLAUDE_MEM_PYTHON_VERSION=3.13 # Python version for chroma-mcp
|
||||
```bash
|
||||
npm run build # Compile TypeScript (hooks + worker)
|
||||
npm run sync-marketplace # Copy to ~/.claude/plugins
|
||||
claude-mem restart # Restart worker
|
||||
npm run worker:restart # Restart worker
|
||||
npm run worker:logs # View worker logs
|
||||
npm run worker:status # Check worker status
|
||||
```
|
||||
|
||||
@@ -48,14 +48,14 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de
|
||||
|
||||
4. Restart worker service:
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
5. Check for port conflicts:
|
||||
```bash
|
||||
# If port 37777 is in use by another service
|
||||
export CLAUDE_MEM_WORKER_PORT=38000
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Theme Toggle Not Persisting
|
||||
@@ -110,7 +110,7 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de
|
||||
|
||||
5. Restart worker and refresh browser:
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Chroma/Python Dependency Issues (v5.0.0+)
|
||||
@@ -225,7 +225,7 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de
|
||||
3. Or use a different port:
|
||||
```bash
|
||||
export CLAUDE_MEM_WORKER_PORT=38000
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
4. Verify new port:
|
||||
@@ -282,9 +282,204 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de
|
||||
|
||||
4. Restart worker:
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Manual Recovery for Stuck Observations
|
||||
|
||||
**Symptoms**: Observations stuck in processing queue after worker crash or restart, no new summaries appearing despite worker running.
|
||||
|
||||
**Background**: As of v5.x, automatic queue recovery on worker startup is disabled. Users must manually trigger recovery to maintain explicit control over reprocessing and prevent unexpected duplicate observations.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
#### Option 1: Use CLI Recovery Tool (Recommended)
|
||||
|
||||
The interactive CLI tool provides the safest and most user-friendly recovery experience:
|
||||
|
||||
```bash
|
||||
# Check queue status and prompt for recovery
|
||||
bun scripts/check-pending-queue.ts
|
||||
|
||||
# Auto-process without prompting
|
||||
bun scripts/check-pending-queue.ts --process
|
||||
|
||||
# Process up to 5 sessions
|
||||
bun scripts/check-pending-queue.ts --process --limit 5
|
||||
```
|
||||
|
||||
**What it does**:
|
||||
- ✅ Checks worker health before proceeding
|
||||
- ✅ Shows detailed queue summary (pending, processing, failed, stuck)
|
||||
- ✅ Groups messages by session with age and status breakdown
|
||||
- ✅ Prompts user to confirm processing (unless `--process` flag used)
|
||||
- ✅ Shows recently processed messages for feedback
|
||||
|
||||
**Interactive Example**:
|
||||
```
|
||||
Worker is healthy ✓
|
||||
|
||||
Queue Summary:
|
||||
Pending: 12 messages
|
||||
Processing: 2 messages (1 stuck)
|
||||
Failed: 0 messages
|
||||
Recently Processed: 5 messages in last 30 minutes
|
||||
|
||||
Sessions with pending work: 3
|
||||
Session 44: 5 pending, 1 processing (age: 2m)
|
||||
Session 45: 4 pending, 1 processing (age: 7m - STUCK)
|
||||
Session 46: 2 pending
|
||||
|
||||
Would you like to process these pending queues? (y/n)
|
||||
```
|
||||
|
||||
#### Option 2: Use HTTP API Directly
|
||||
|
||||
For automation or scripting scenarios:
|
||||
|
||||
1. **Check queue status**:
|
||||
```bash
|
||||
curl http://localhost:37777/api/pending-queue
|
||||
```
|
||||
|
||||
Response shows:
|
||||
- `queue.totalPending`: Messages waiting to process
|
||||
- `queue.totalProcessing`: Messages currently processing
|
||||
- `queue.stuckCount`: Processing messages >5 minutes old
|
||||
- `sessionsWithPendingWork`: Session IDs needing recovery
|
||||
|
||||
2. **Trigger manual recovery**:
|
||||
```bash
|
||||
curl -X POST http://localhost:37777/api/pending-queue/process \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"sessionLimit": 10}'
|
||||
```
|
||||
|
||||
Response includes:
|
||||
- `totalPendingSessions`: Total sessions with pending messages
|
||||
- `sessionsStarted`: Number of sessions we started processing
|
||||
- `sessionsSkipped`: Sessions already processing (not restarted)
|
||||
- `startedSessionIds`: Database IDs of sessions started
|
||||
|
||||
#### Understanding Queue States
|
||||
|
||||
Messages progress through these states:
|
||||
|
||||
1. **pending** - Queued, waiting to process
|
||||
2. **processing** - Currently being processed by SDK agent
|
||||
3. **processed** - Completed successfully
|
||||
4. **failed** - Failed after 3 retry attempts
|
||||
|
||||
**Stuck Detection**: Messages in `processing` state for >5 minutes are considered stuck and automatically reset to `pending` on worker startup (but not automatically reprocessed).
|
||||
|
||||
#### Recovery Strategy
|
||||
|
||||
**When to use manual recovery**:
|
||||
- After worker crashes or unexpected restarts
|
||||
- When observations appear saved but no summaries generated
|
||||
- When queue status shows stuck messages (processing >5 minutes)
|
||||
- After system crashes or forced shutdowns
|
||||
|
||||
**Best practices**:
|
||||
1. Always check queue status before triggering recovery
|
||||
2. Use the CLI tool for interactive sessions (provides feedback)
|
||||
3. Use the HTTP API for automation/scripting
|
||||
4. Start with a low session limit (5-10) to avoid overwhelming the worker
|
||||
5. Monitor worker logs during recovery: `npm run worker:logs`
|
||||
6. Check recently processed messages to confirm recovery worked
|
||||
|
||||
#### Troubleshooting Recovery Issues
|
||||
|
||||
If recovery fails or messages remain stuck:
|
||||
|
||||
1. **Verify worker is healthy**:
|
||||
```bash
|
||||
curl http://localhost:37777/health
|
||||
# Should return: {"status":"ok","uptime":12345,"port":37777}
|
||||
```
|
||||
|
||||
2. **Check database for corruption**:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
|
||||
3. **View stuck messages directly**:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT id, session_db_id, status, retry_count,
|
||||
(strftime('%s', 'now') * 1000 - started_processing_at_epoch) / 60000 as age_minutes
|
||||
FROM pending_messages
|
||||
WHERE status = 'processing'
|
||||
ORDER BY started_processing_at_epoch;
|
||||
"
|
||||
```
|
||||
|
||||
4. **Force reset stuck messages** (nuclear option):
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
UPDATE pending_messages
|
||||
SET status = 'pending', started_processing_at_epoch = NULL
|
||||
WHERE status = 'processing';
|
||||
"
|
||||
```
|
||||
|
||||
Then trigger recovery:
|
||||
```bash
|
||||
bun scripts/check-pending-queue.ts --process
|
||||
```
|
||||
|
||||
5. **Check worker logs for SDK errors**:
|
||||
```bash
|
||||
npm run worker:logs | grep -i error
|
||||
```
|
||||
|
||||
#### Understanding the Queue Table
|
||||
|
||||
The `pending_messages` table tracks all messages with these key fields:
|
||||
|
||||
```sql
|
||||
CREATE TABLE pending_messages (
|
||||
id INTEGER PRIMARY KEY,
|
||||
session_db_id INTEGER, -- Foreign key to sdk_sessions
|
||||
claude_session_id TEXT, -- Claude session ID
|
||||
message_type TEXT, -- 'observation' | 'summarize'
|
||||
status TEXT, -- 'pending' | 'processing' | 'processed' | 'failed'
|
||||
retry_count INTEGER, -- Current retry attempt (max: 3)
|
||||
created_at_epoch INTEGER, -- When message was queued
|
||||
started_processing_at_epoch INTEGER, -- When marked 'processing'
|
||||
completed_at_epoch INTEGER -- When completed/failed
|
||||
)
|
||||
```
|
||||
|
||||
**Query examples**:
|
||||
|
||||
```bash
|
||||
# Count messages by status
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT status, COUNT(*)
|
||||
FROM pending_messages
|
||||
GROUP BY status;
|
||||
"
|
||||
|
||||
# Find sessions with pending work
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT session_db_id, COUNT(*) as pending_count
|
||||
FROM pending_messages
|
||||
WHERE status IN ('pending', 'processing')
|
||||
GROUP BY session_db_id;
|
||||
"
|
||||
|
||||
# View recent failures
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT id, session_db_id, message_type, retry_count,
|
||||
datetime(completed_at_epoch/1000, 'unixepoch') as failed_at
|
||||
FROM pending_messages
|
||||
WHERE status = 'failed'
|
||||
ORDER BY completed_at_epoch DESC
|
||||
LIMIT 10;
|
||||
"
|
||||
```
|
||||
|
||||
## Hook Issues
|
||||
|
||||
### Hooks Not Firing
|
||||
@@ -644,7 +839,7 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de
|
||||
|
||||
2. Restart worker:
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
3. Clean up old data (see "Database Too Large" above)
|
||||
@@ -721,7 +916,7 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de
|
||||
|
||||
```bash
|
||||
export DEBUG=claude-mem:*
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
@@ -781,7 +976,7 @@ SELECT created_at, tool_name FROM observations ORDER BY created_at DESC LIMIT 10
|
||||
|
||||
**Cause**: Worker not running or port mismatch.
|
||||
|
||||
**Solution**: Restart worker with `claude-mem restart`.
|
||||
**Solution**: Restart worker with `npm run worker:restart`.
|
||||
|
||||
### "Database is locked"
|
||||
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
---
|
||||
title: "Gemini Provider"
|
||||
description: "Use Google's Gemini API as an alternative to Claude for observation extraction"
|
||||
---
|
||||
|
||||
# Gemini Provider
|
||||
|
||||
Claude-mem supports Google's Gemini API as an alternative to the Claude Agent SDK for extracting observations from your sessions. This can significantly reduce costs since Gemini offers a generous free tier.
|
||||
|
||||
<Warning>
|
||||
**Free Tier Rate Limits**: Without billing enabled, Gemini has strict rate limits (5-10 RPM). Enable billing on your Google Cloud project to unlock 1000-4000 RPM while still using the free quota.
|
||||
</Warning>
|
||||
|
||||
## Why Use Gemini?
|
||||
|
||||
- **Cost savings**: The free tier covers most individual usage patterns
|
||||
- **Same quality**: Gemini extracts observations using the same XML format as Claude
|
||||
- **Seamless fallback**: Automatically falls back to Claude if Gemini is unavailable
|
||||
- **Hot-swappable**: Switch providers without restarting the worker
|
||||
|
||||
## Getting a Free API Key
|
||||
|
||||
1. Go to the [Google AI Studio API Key page](https://aistudio.google.com/app/apikey)
|
||||
2. Sign in with your Google account
|
||||
3. Accept the Terms of Service and privacy policies
|
||||
4. Click the **Create API key** button
|
||||
5. Choose a Google Cloud project or create a new one
|
||||
6. Copy and securely store the generated API key
|
||||
|
||||
<Tip>
|
||||
**No billing required** to get started, but we recommend enabling billing to unlock higher rate limits (1000-4000 RPM vs 5-10 RPM) while still using the free quota.
|
||||
</Tip>
|
||||
|
||||
## Configuration
|
||||
|
||||
### Settings
|
||||
|
||||
| Setting | Values | Default | Description |
|
||||
|---------|--------|---------|-------------|
|
||||
| `CLAUDE_MEM_PROVIDER` | `claude`, `gemini` | `claude` | AI provider for observation extraction |
|
||||
| `CLAUDE_MEM_GEMINI_API_KEY` | string | — | Your Gemini API key |
|
||||
| `CLAUDE_MEM_GEMINI_MODEL` | `gemini-2.5-flash-lite`, `gemini-2.5-flash`, `gemini-3-flash` | `gemini-2.5-flash-lite` | Gemini model to use |
|
||||
| `CLAUDE_MEM_GEMINI_BILLING_ENABLED` | `true`, `false` | `false` | Skip rate limiting if billing is enabled on Google Cloud |
|
||||
|
||||
### Using the Settings UI
|
||||
|
||||
1. Open the viewer at http://localhost:37777
|
||||
2. Click the **gear icon** to open Settings
|
||||
3. Under **AI Provider**, select **Gemini**
|
||||
4. Enter your Gemini API key
|
||||
5. Optionally select a different model
|
||||
|
||||
Settings are applied immediately—no restart required.
|
||||
|
||||
### Manual Configuration
|
||||
|
||||
Edit `~/.claude-mem/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"CLAUDE_MEM_PROVIDER": "gemini",
|
||||
"CLAUDE_MEM_GEMINI_API_KEY": "your-api-key-here",
|
||||
"CLAUDE_MEM_GEMINI_MODEL": "gemini-2.5-flash-lite",
|
||||
"CLAUDE_MEM_GEMINI_BILLING_ENABLED": "true"
|
||||
}
|
||||
```
|
||||
|
||||
Alternatively, set the API key via environment variable:
|
||||
|
||||
```bash
|
||||
export GEMINI_API_KEY="your-api-key-here"
|
||||
```
|
||||
|
||||
The settings file takes precedence over the environment variable.
|
||||
|
||||
## Available Models
|
||||
|
||||
| Model | Free Tier RPM | Notes |
|
||||
|-------|--------------|-------|
|
||||
| `gemini-2.5-flash-lite` | 10 | Default, recommended for free tier (highest RPM) |
|
||||
| `gemini-2.5-flash` | 5 | Higher capability, lower rate limit |
|
||||
| `gemini-3-flash` | 5 | Latest model, lower rate limit |
|
||||
|
||||
## Provider Switching
|
||||
|
||||
You can switch between Claude and Gemini at any time:
|
||||
|
||||
- **No restart required**: Changes take effect on the next observation
|
||||
- **Conversation history preserved**: When switching mid-session, the new provider sees the full conversation context
|
||||
- **Seamless transition**: Both providers use the same observation format
|
||||
|
||||
### Switching via UI
|
||||
|
||||
1. Open Settings in the viewer
|
||||
2. Change the **AI Provider** dropdown
|
||||
3. The next observation will use the new provider
|
||||
|
||||
### Switching via Settings File
|
||||
|
||||
```json
|
||||
{
|
||||
"CLAUDE_MEM_PROVIDER": "gemini"
|
||||
}
|
||||
```
|
||||
|
||||
## Fallback Behavior
|
||||
|
||||
If Gemini is selected but encounters errors, claude-mem automatically falls back to the Claude Agent SDK:
|
||||
|
||||
**Triggers fallback:**
|
||||
- Rate limiting (HTTP 429)
|
||||
- Server errors (HTTP 5xx)
|
||||
- Network issues (connection refused, timeout)
|
||||
|
||||
**Does not trigger fallback:**
|
||||
- Missing API key (logs warning, uses Claude from start)
|
||||
- Invalid API key (fails with error)
|
||||
|
||||
When fallback occurs:
|
||||
1. A warning is logged
|
||||
2. Any in-progress messages are reset to pending
|
||||
3. Claude SDK takes over with the full conversation context
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Gemini API key not configured"
|
||||
|
||||
Either:
|
||||
- Set `CLAUDE_MEM_GEMINI_API_KEY` in `~/.claude-mem/settings.json`, or
|
||||
- Set the `GEMINI_API_KEY` environment variable
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
Google has two rate limit tiers for free usage:
|
||||
|
||||
**Without billing (API key only):**
|
||||
|
||||
| Model | RPM | TPM |
|
||||
|-------|-----|-----|
|
||||
| gemini-2.5-flash-lite | 10 | 250K |
|
||||
| gemini-2.5-flash | 5 | 250K |
|
||||
| gemini-3-flash | 5 | 250K |
|
||||
|
||||
Claude-mem enforces these limits automatically with built-in delays between requests. Processing may be slower but stays within limits.
|
||||
|
||||
**With billing enabled (still free tier):**
|
||||
|
||||
| Model | RPM | TPM |
|
||||
|-------|-----|-----|
|
||||
| gemini-2.5-flash-lite | 4,000 | 4M |
|
||||
| gemini-2.5-flash | 1,000 | 1M |
|
||||
| gemini-3-flash | 1,000 | 1M |
|
||||
|
||||
<Tip>
|
||||
**Recommended**: Enable billing on your Google Cloud project to unlock much higher rate limits. You won't be charged unless you exceed the generous free quota. This allows claude-mem to process observations instantly instead of waiting between requests.
|
||||
</Tip>
|
||||
|
||||
If you hit rate limits:
|
||||
- Claude-mem automatically falls back to Claude SDK
|
||||
- Or switch back to Claude as your primary provider
|
||||
|
||||
### Observation Quality
|
||||
|
||||
If observations seem lower quality with Gemini:
|
||||
- Note that Claude typically produces slightly higher quality observations
|
||||
- Consider using Gemini for cost savings and Claude for important projects
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Configuration](../configuration) - Full settings reference
|
||||
- [Getting Started](getting-started) - Basic usage guide
|
||||
- [Troubleshooting](../troubleshooting) - Common issues
|
||||
@@ -86,7 +86,7 @@ npm run worker:start
|
||||
npm run worker:stop
|
||||
|
||||
# Restart worker service
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
|
||||
# View worker logs
|
||||
npm run worker:logs
|
||||
|
||||
@@ -0,0 +1,450 @@
|
||||
---
|
||||
title: "Manual Recovery"
|
||||
description: "Recover stuck observations after worker crashes or restarts"
|
||||
---
|
||||
|
||||
# Manual Recovery Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Claude-mem's manual recovery system helps you recover observations that get stuck in the processing queue after worker crashes, system restarts, or unexpected shutdowns.
|
||||
|
||||
**Key Change in v5.x**: Automatic recovery on worker startup is now disabled. This gives you explicit control over when reprocessing happens, preventing unexpected duplicate observations.
|
||||
|
||||
## When Do You Need Manual Recovery?
|
||||
|
||||
You should trigger manual recovery when:
|
||||
|
||||
- **Worker crashed or restarted** - Observations were queued but worker stopped before processing
|
||||
- **No new summaries appearing** - Observations are being saved but not processed into summaries
|
||||
- **Stuck messages detected** - Messages showing as "processing" for >5 minutes
|
||||
- **System crashes** - Unexpected shutdowns left messages in incomplete states
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Using the CLI Tool (Recommended)
|
||||
|
||||
The interactive CLI tool is the safest and easiest way to recover stuck observations:
|
||||
|
||||
```bash
|
||||
# Check status and prompt for recovery
|
||||
bun scripts/check-pending-queue.ts
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Check worker health
|
||||
2. Show queue summary (pending, processing, failed, stuck counts)
|
||||
3. Display sessions with pending work
|
||||
4. Prompt you to confirm recovery
|
||||
5. Show recently processed messages for feedback
|
||||
|
||||
### Auto-Process Without Prompts
|
||||
|
||||
For scripting or when you're confident recovery is needed:
|
||||
|
||||
```bash
|
||||
# Auto-process without prompting
|
||||
bun scripts/check-pending-queue.ts --process
|
||||
|
||||
# Limit to 5 sessions
|
||||
bun scripts/check-pending-queue.ts --process --limit 5
|
||||
```
|
||||
|
||||
## Understanding Queue States
|
||||
|
||||
Messages progress through these lifecycle states:
|
||||
|
||||
1. **pending** → Queued, waiting to process
|
||||
2. **processing** → Currently being processed by SDK agent
|
||||
3. **processed** → Completed successfully
|
||||
4. **failed** → Failed after 3 retry attempts
|
||||
|
||||
### Stuck Detection
|
||||
|
||||
Messages in `processing` state for **>5 minutes** are considered stuck:
|
||||
|
||||
- They're automatically reset to `pending` on worker startup
|
||||
- They're NOT automatically reprocessed (requires manual trigger)
|
||||
- They appear in the `stuckCount` field when checking queue status
|
||||
|
||||
## Recovery Methods
|
||||
|
||||
### Method 1: Interactive CLI Tool
|
||||
|
||||
**Best for**: Regular users, interactive sessions, when you want visibility into what's happening
|
||||
|
||||
```bash
|
||||
bun scripts/check-pending-queue.ts
|
||||
```
|
||||
|
||||
**Example Output**:
|
||||
```
|
||||
Checking worker health...
|
||||
Worker is healthy ✓
|
||||
|
||||
Queue Summary:
|
||||
Pending: 12 messages
|
||||
Processing: 2 messages (1 stuck)
|
||||
Failed: 0 messages
|
||||
Recently Processed: 5 messages in last 30 minutes
|
||||
|
||||
Sessions with pending work: 3
|
||||
Session 44: 5 pending, 1 processing (age: 2m)
|
||||
Session 45: 4 pending, 1 processing (age: 7m - STUCK)
|
||||
Session 46: 2 pending
|
||||
|
||||
Would you like to process these pending queues? (y/n)
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- ✅ Pre-flight health check (verifies worker is running)
|
||||
- ✅ Detailed queue breakdown by session
|
||||
- ✅ Age tracking for stuck detection
|
||||
- ✅ Confirmation prompt (prevents accidental reprocessing)
|
||||
- ✅ Non-interactive mode with `--process` flag
|
||||
- ✅ Session limit control with `--limit N`
|
||||
|
||||
### Method 2: HTTP API
|
||||
|
||||
**Best for**: Automation, scripting, integration with monitoring systems
|
||||
|
||||
#### Check Queue Status
|
||||
|
||||
```bash
|
||||
curl http://localhost:37777/api/pending-queue
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"queue": {
|
||||
"messages": [
|
||||
{
|
||||
"id": 123,
|
||||
"session_db_id": 45,
|
||||
"claude_session_id": "abc123",
|
||||
"message_type": "observation",
|
||||
"status": "pending",
|
||||
"retry_count": 0,
|
||||
"created_at_epoch": 1730886600000
|
||||
}
|
||||
],
|
||||
"totalPending": 12,
|
||||
"totalProcessing": 2,
|
||||
"totalFailed": 0,
|
||||
"stuckCount": 1
|
||||
},
|
||||
"recentlyProcessed": [...],
|
||||
"sessionsWithPendingWork": [44, 45, 46]
|
||||
}
|
||||
```
|
||||
|
||||
**Key Fields**:
|
||||
- `totalPending` - Messages waiting to process
|
||||
- `totalProcessing` - Messages currently processing
|
||||
- `stuckCount` - Processing messages >5 minutes old
|
||||
- `sessionsWithPendingWork` - Session IDs needing recovery
|
||||
|
||||
#### Trigger Recovery
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:37777/api/pending-queue/process \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"sessionLimit": 10}'
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"totalPendingSessions": 15,
|
||||
"sessionsStarted": 10,
|
||||
"sessionsSkipped": 2,
|
||||
"startedSessionIds": [44, 45, 46, 47, 48, 49, 50, 51, 52, 53]
|
||||
}
|
||||
```
|
||||
|
||||
**Response Fields**:
|
||||
- `totalPendingSessions` - Total sessions with pending messages in database
|
||||
- `sessionsStarted` - Sessions we started processing this request
|
||||
- `sessionsSkipped` - Sessions already processing (prevents duplicate agents)
|
||||
- `startedSessionIds` - Database IDs of sessions we started
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Always Check Before Recovery
|
||||
|
||||
```bash
|
||||
# Check queue status first
|
||||
curl http://localhost:37777/api/pending-queue
|
||||
|
||||
# Or use CLI tool which checks automatically
|
||||
bun scripts/check-pending-queue.ts
|
||||
```
|
||||
|
||||
### 2. Start with Low Session Limits
|
||||
|
||||
```bash
|
||||
# Process only 5 sessions at a time
|
||||
bun scripts/check-pending-queue.ts --process --limit 5
|
||||
```
|
||||
|
||||
This prevents overwhelming the worker with too many concurrent SDK agents.
|
||||
|
||||
### 3. Monitor During Recovery
|
||||
|
||||
Watch worker logs while recovery runs:
|
||||
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
Look for:
|
||||
- SDK agent starts: `Starting SDK agent for session...`
|
||||
- Processing completions: `Processed observation...`
|
||||
- Errors: `ERROR` or `Failed to process...`
|
||||
|
||||
### 4. Verify Recovery Success
|
||||
|
||||
Check recently processed messages:
|
||||
|
||||
```bash
|
||||
curl http://localhost:37777/api/pending-queue | jq '.recentlyProcessed'
|
||||
```
|
||||
|
||||
Or use the CLI tool which shows this automatically.
|
||||
|
||||
### 5. Handle Failed Messages
|
||||
|
||||
Messages that fail 3 times are marked `failed` and won't auto-retry:
|
||||
|
||||
```bash
|
||||
# View failed messages
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT id, session_db_id, message_type, retry_count
|
||||
FROM pending_messages
|
||||
WHERE status = 'failed'
|
||||
ORDER BY completed_at_epoch DESC;
|
||||
"
|
||||
```
|
||||
|
||||
You can manually reset them if needed:
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
UPDATE pending_messages
|
||||
SET status = 'pending', retry_count = 0
|
||||
WHERE status = 'failed';
|
||||
"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Recovery Not Working
|
||||
|
||||
**Symptom**: Triggered recovery but messages still pending
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. **Verify worker health**:
|
||||
```bash
|
||||
curl http://localhost:37777/health
|
||||
```
|
||||
|
||||
2. **Check worker logs for errors**:
|
||||
```bash
|
||||
npm run worker:logs | grep -i error
|
||||
```
|
||||
|
||||
3. **Restart worker**:
|
||||
```bash
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
4. **Check database integrity**:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
|
||||
### Messages Stuck Forever
|
||||
|
||||
**Symptom**: Messages show as "processing" for hours
|
||||
|
||||
**Solution**: Force reset stuck messages
|
||||
|
||||
```bash
|
||||
# Reset all stuck messages to pending
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
UPDATE pending_messages
|
||||
SET status = 'pending', started_processing_at_epoch = NULL
|
||||
WHERE status = 'processing';
|
||||
"
|
||||
|
||||
# Then trigger recovery
|
||||
bun scripts/check-pending-queue.ts --process
|
||||
```
|
||||
|
||||
### Worker Crashes During Recovery
|
||||
|
||||
**Symptom**: Worker stops while processing recovered messages
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. **Check available memory**:
|
||||
```bash
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
2. **Reduce session limit**:
|
||||
```bash
|
||||
bun scripts/check-pending-queue.ts --process --limit 3
|
||||
```
|
||||
|
||||
3. **Check for SDK errors in logs**:
|
||||
```bash
|
||||
npm run worker:logs | grep -i "sdk"
|
||||
```
|
||||
|
||||
4. **Increase worker memory** (if using custom runner):
|
||||
```bash
|
||||
export NODE_OPTIONS="--max-old-space-size=4096"
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Direct Database Inspection
|
||||
|
||||
View all pending messages:
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
id,
|
||||
session_db_id,
|
||||
message_type,
|
||||
status,
|
||||
retry_count,
|
||||
datetime(created_at_epoch/1000, 'unixepoch') as created_at,
|
||||
datetime(started_processing_at_epoch/1000, 'unixepoch') as started_at,
|
||||
CAST((strftime('%s', 'now') * 1000 - started_processing_at_epoch) / 60000 AS INTEGER) as age_minutes
|
||||
FROM pending_messages
|
||||
WHERE status IN ('pending', 'processing')
|
||||
ORDER BY created_at_epoch;
|
||||
"
|
||||
```
|
||||
|
||||
### Count Messages by Status
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT status, COUNT(*) as count
|
||||
FROM pending_messages
|
||||
GROUP BY status;
|
||||
"
|
||||
```
|
||||
|
||||
### Find Sessions with Pending Work
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
session_db_id,
|
||||
COUNT(*) as pending_count,
|
||||
GROUP_CONCAT(message_type) as message_types
|
||||
FROM pending_messages
|
||||
WHERE status IN ('pending', 'processing')
|
||||
GROUP BY session_db_id;
|
||||
"
|
||||
```
|
||||
|
||||
### View Recent Failures
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
id,
|
||||
session_db_id,
|
||||
message_type,
|
||||
retry_count,
|
||||
datetime(completed_at_epoch/1000, 'unixepoch') as failed_at
|
||||
FROM pending_messages
|
||||
WHERE status = 'failed'
|
||||
ORDER BY completed_at_epoch DESC
|
||||
LIMIT 10;
|
||||
"
|
||||
```
|
||||
|
||||
## Integration Examples
|
||||
|
||||
### Cron Job for Automatic Recovery
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Run every hour to process stuck queues
|
||||
|
||||
# Check if worker is healthy
|
||||
if curl -f http://localhost:37777/health > /dev/null 2>&1; then
|
||||
# Auto-process up to 5 sessions
|
||||
bun scripts/check-pending-queue.ts --process --limit 5
|
||||
else
|
||||
echo "Worker not healthy, skipping recovery"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
### Monitoring Script
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Alert if stuck count exceeds threshold
|
||||
|
||||
STUCK_COUNT=$(curl -s http://localhost:37777/api/pending-queue | jq '.queue.stuckCount')
|
||||
|
||||
if [ "$STUCK_COUNT" -gt 5 ]; then
|
||||
echo "WARNING: $STUCK_COUNT stuck messages detected"
|
||||
# Send alert (email, Slack, etc.)
|
||||
fi
|
||||
```
|
||||
|
||||
### Pre-Shutdown Recovery
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Process pending queues before system shutdown
|
||||
|
||||
echo "Processing pending queues before shutdown..."
|
||||
bun scripts/check-pending-queue.ts --process --limit 20
|
||||
|
||||
echo "Waiting for processing to complete..."
|
||||
sleep 10
|
||||
|
||||
echo "Stopping worker..."
|
||||
claude-mem stop
|
||||
```
|
||||
|
||||
## Migration Note
|
||||
|
||||
If you're upgrading from v4.x to v5.x:
|
||||
|
||||
**v4.x Behavior** (Automatic Recovery):
|
||||
- Worker automatically recovered stuck messages on startup
|
||||
- No user control over reprocessing timing
|
||||
|
||||
**v5.x Behavior** (Manual Recovery):
|
||||
- Stuck messages detected but NOT automatically reprocessed
|
||||
- User must explicitly trigger recovery via CLI or API
|
||||
- Prevents unexpected duplicate observations
|
||||
- Provides explicit control over when processing happens
|
||||
|
||||
**Migration Steps**:
|
||||
1. Upgrade to v5.x
|
||||
2. Check for stuck messages: `bun scripts/check-pending-queue.ts`
|
||||
3. Process if needed: `bun scripts/check-pending-queue.ts --process`
|
||||
4. Add recovery to your workflow (cron job, pre-shutdown script, etc.)
|
||||
|
||||
## See Also
|
||||
|
||||
- [Worker Service Architecture](../architecture/worker-service) - Technical details on queue processing
|
||||
- [Troubleshooting - Manual Recovery](../troubleshooting#manual-recovery-for-stuck-observations) - Common issues and solutions
|
||||
- [Database Schema](../architecture/database) - Pending messages table structure
|
||||
@@ -0,0 +1,320 @@
|
||||
---
|
||||
title: "OpenRouter Provider"
|
||||
description: "Access 100+ AI models through OpenRouter's unified API, including free models for cost-effective observation extraction"
|
||||
---
|
||||
|
||||
# OpenRouter Provider
|
||||
|
||||
Claude-mem supports [OpenRouter](https://openrouter.ai) as an alternative provider for observation extraction. OpenRouter provides a unified API to access 100+ models from different providers including Google, Meta, Mistral, DeepSeek, and many others—often with generous free tiers.
|
||||
|
||||
<Tip>
|
||||
**Free Models Available**: OpenRouter offers several completely free models, making it an excellent choice for reducing observation extraction costs to zero while maintaining quality.
|
||||
</Tip>
|
||||
|
||||
## Why Use OpenRouter?
|
||||
|
||||
- **Access to 100+ models**: Choose from models across multiple providers through one API
|
||||
- **Free tier options**: Several high-quality models are completely free to use
|
||||
- **Cost flexibility**: Pay-as-you-go pricing on premium models with no commitments
|
||||
- **Seamless fallback**: Automatically falls back to Claude if OpenRouter is unavailable
|
||||
- **Hot-swappable**: Switch providers without restarting the worker
|
||||
- **Multi-turn conversations**: Full conversation history maintained across API calls
|
||||
|
||||
## Free Models on OpenRouter
|
||||
|
||||
OpenRouter actively supports democratizing AI access by offering free models. These are production-ready models suitable for observation extraction.
|
||||
|
||||
### Featured Free Models
|
||||
|
||||
| Model | ID | Parameters | Context | Best For |
|
||||
|-------|------|------------|---------|----------|
|
||||
| **Xiaomi MiMo-V2-Flash** | `xiaomi/mimo-v2-flash:free` | 309B (15B active, MoE) | 256K | Reasoning, coding, agents |
|
||||
| **Gemini 2.0 Flash** | `google/gemini-2.0-flash-exp:free` | — | 1M | General purpose |
|
||||
| **Gemini 2.5 Flash** | `google/gemini-2.5-flash-preview:free` | — | 1M | Latest capabilities |
|
||||
| **DeepSeek R1** | `deepseek/deepseek-r1:free` | 671B | 64K | Reasoning, analysis |
|
||||
| **Llama 3.1 70B** | `meta-llama/llama-3.1-70b-instruct:free` | 70B | 128K | General purpose |
|
||||
| **Llama 3.1 8B** | `meta-llama/llama-3.1-8b-instruct:free` | 8B | 128K | Fast, lightweight |
|
||||
| **Mistral Nemo** | `mistralai/mistral-nemo:free` | 12B | 128K | Efficient performance |
|
||||
|
||||
<Note>
|
||||
**Default Model**: Claude-mem uses `xiaomi/mimo-v2-flash:free` by default—a 309B parameter mixture-of-experts model that ranks #1 on SWE-bench Verified and excels at coding and reasoning tasks.
|
||||
</Note>
|
||||
|
||||
### Free Model Considerations
|
||||
|
||||
- **Rate limits**: Free models may have stricter rate limits than paid models
|
||||
- **Availability**: Free capacity depends on provider partnerships and demand
|
||||
- **Queue times**: During peak usage, requests may be queued briefly
|
||||
- **Max tokens**: Most free models support 65,536 completion tokens
|
||||
|
||||
All free models support:
|
||||
- Tool use and function calling
|
||||
- Temperature and sampling controls
|
||||
- Stop sequences
|
||||
- Streaming responses
|
||||
|
||||
## Getting an API Key
|
||||
|
||||
1. Go to [OpenRouter](https://openrouter.ai)
|
||||
2. Sign in with Google, GitHub, or email
|
||||
3. Navigate to [API Keys](https://openrouter.ai/keys)
|
||||
4. Click **Create Key**
|
||||
5. Copy and securely store your API key
|
||||
|
||||
<Tip>
|
||||
**Free to start**: No credit card required to create an account or use free models. Add credits only if you want to use premium models.
|
||||
</Tip>
|
||||
|
||||
## Configuration
|
||||
|
||||
### Settings
|
||||
|
||||
| Setting | Values | Default | Description |
|
||||
|---------|--------|---------|-------------|
|
||||
| `CLAUDE_MEM_PROVIDER` | `claude`, `gemini`, `openrouter` | `claude` | AI provider for observation extraction |
|
||||
| `CLAUDE_MEM_OPENROUTER_API_KEY` | string | — | Your OpenRouter API key |
|
||||
| `CLAUDE_MEM_OPENROUTER_MODEL` | string | `xiaomi/mimo-v2-flash:free` | Model identifier (see list above) |
|
||||
| `CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES` | number | `20` | Max messages in conversation history |
|
||||
| `CLAUDE_MEM_OPENROUTER_MAX_TOKENS` | number | `100000` | Token budget safety limit |
|
||||
| `CLAUDE_MEM_OPENROUTER_SITE_URL` | string | — | Optional: URL for analytics attribution |
|
||||
| `CLAUDE_MEM_OPENROUTER_APP_NAME` | string | `claude-mem` | Optional: App name for analytics |
|
||||
|
||||
### Using the Settings UI
|
||||
|
||||
1. Open the viewer at http://localhost:37777
|
||||
2. Click the **gear icon** to open Settings
|
||||
3. Under **AI Provider**, select **OpenRouter**
|
||||
4. Enter your OpenRouter API key
|
||||
5. Optionally select a different model
|
||||
|
||||
Settings are applied immediately—no restart required.
|
||||
|
||||
### Manual Configuration
|
||||
|
||||
Edit `~/.claude-mem/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"CLAUDE_MEM_PROVIDER": "openrouter",
|
||||
"CLAUDE_MEM_OPENROUTER_API_KEY": "sk-or-v1-your-key-here",
|
||||
"CLAUDE_MEM_OPENROUTER_MODEL": "xiaomi/mimo-v2-flash:free"
|
||||
}
|
||||
```
|
||||
|
||||
Alternatively, set the API key via environment variable:
|
||||
|
||||
```bash
|
||||
export OPENROUTER_API_KEY="sk-or-v1-your-key-here"
|
||||
```
|
||||
|
||||
The settings file takes precedence over the environment variable.
|
||||
|
||||
## Model Selection Guide
|
||||
|
||||
### For Free Usage (No Cost)
|
||||
|
||||
**Recommended**: `xiaomi/mimo-v2-flash:free`
|
||||
- Best-in-class performance on coding benchmarks
|
||||
- 256K context window handles large observations
|
||||
- 65K max completion tokens
|
||||
- Mixture-of-experts architecture (15B active parameters)
|
||||
|
||||
**Alternatives**:
|
||||
- `google/gemini-2.0-flash-exp:free` - 1M context, Google's flagship
|
||||
- `deepseek/deepseek-r1:free` - Excellent reasoning capabilities
|
||||
- `meta-llama/llama-3.1-70b-instruct:free` - Strong general purpose
|
||||
|
||||
### For Paid Usage (Higher Quality/Speed)
|
||||
|
||||
| Model | Price (per 1M tokens) | Best For |
|
||||
|-------|----------------------|----------|
|
||||
| `anthropic/claude-3.5-sonnet` | $3 in / $15 out | Highest quality observations |
|
||||
| `google/gemini-2.0-flash` | $0.075 in / $0.30 out | Fast, cost-effective |
|
||||
| `openai/gpt-4o` | $2.50 in / $10 out | GPT-4 quality |
|
||||
|
||||
## Context Window Management
|
||||
|
||||
OpenRouter agent implements intelligent context management to prevent runaway costs:
|
||||
|
||||
### Automatic Truncation
|
||||
|
||||
The agent uses a sliding window strategy:
|
||||
1. Checks if message count exceeds `MAX_CONTEXT_MESSAGES` (default: 20)
|
||||
2. Checks if estimated tokens exceed `MAX_TOKENS` (default: 100,000)
|
||||
3. If limits exceeded, keeps most recent messages only
|
||||
4. Logs warnings with dropped message counts
|
||||
|
||||
### Token Estimation
|
||||
|
||||
- Conservative estimate: 1 token ≈ 4 characters
|
||||
- Used for proactive context management
|
||||
- Actual usage logged from API response
|
||||
|
||||
### Cost Tracking
|
||||
|
||||
Logs include detailed usage information:
|
||||
|
||||
```
|
||||
OpenRouter API usage: {
|
||||
model: "xiaomi/mimo-v2-flash:free",
|
||||
inputTokens: 2500,
|
||||
outputTokens: 1200,
|
||||
totalTokens: 3700,
|
||||
estimatedCostUSD: "0.00",
|
||||
messagesInContext: 8
|
||||
}
|
||||
```
|
||||
|
||||
## Provider Switching
|
||||
|
||||
You can switch between providers at any time:
|
||||
|
||||
- **No restart required**: Changes take effect on the next observation
|
||||
- **Conversation history preserved**: When switching mid-session, the new provider sees the full conversation context
|
||||
- **Seamless transition**: All providers use the same observation format
|
||||
|
||||
### Switching via UI
|
||||
|
||||
1. Open Settings in the viewer
|
||||
2. Change the **AI Provider** dropdown
|
||||
3. The next observation will use the new provider
|
||||
|
||||
### Switching via Settings File
|
||||
|
||||
```json
|
||||
{
|
||||
"CLAUDE_MEM_PROVIDER": "openrouter"
|
||||
}
|
||||
```
|
||||
|
||||
## Fallback Behavior
|
||||
|
||||
If OpenRouter encounters errors, claude-mem automatically falls back to the Claude Agent SDK:
|
||||
|
||||
**Triggers fallback:**
|
||||
- Rate limiting (HTTP 429)
|
||||
- Server errors (HTTP 500, 502, 503)
|
||||
- Network issues (connection refused, timeout)
|
||||
- Generic fetch failures
|
||||
|
||||
**Does not trigger fallback:**
|
||||
- Missing API key (logs warning, uses Claude from start)
|
||||
- Invalid API key (fails with error)
|
||||
|
||||
When fallback occurs:
|
||||
1. A warning is logged
|
||||
2. Any in-progress messages are reset to pending
|
||||
3. Claude SDK takes over with the full conversation context
|
||||
|
||||
<Note>
|
||||
**Fallback is transparent**: Your observations continue processing without interruption. The fallback preserves all conversation context.
|
||||
</Note>
|
||||
|
||||
## Multi-Turn Conversation Support
|
||||
|
||||
OpenRouter agent maintains full conversation history across API calls:
|
||||
|
||||
```
|
||||
Session Created
|
||||
↓
|
||||
Load Pending Messages (observations from queue)
|
||||
↓
|
||||
For each message:
|
||||
→ Add to conversation history
|
||||
→ Call OpenRouter API with FULL history
|
||||
→ Parse XML response
|
||||
→ Store observations in database
|
||||
→ Sync to Chroma vector DB
|
||||
↓
|
||||
Session complete
|
||||
```
|
||||
|
||||
This enables:
|
||||
- Coherent multi-turn exchanges
|
||||
- Context preservation across observations
|
||||
- Seamless provider switching mid-session
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "OpenRouter API key not configured"
|
||||
|
||||
Either:
|
||||
- Set `CLAUDE_MEM_OPENROUTER_API_KEY` in `~/.claude-mem/settings.json`, or
|
||||
- Set the `OPENROUTER_API_KEY` environment variable
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
Free models may have rate limits during peak usage. If you hit rate limits:
|
||||
- Claude-mem automatically falls back to Claude SDK
|
||||
- Consider switching to a different free model
|
||||
- Add credits for premium model access
|
||||
|
||||
### Model Not Found
|
||||
|
||||
Verify the model ID is correct:
|
||||
- Check [OpenRouter Models](https://openrouter.ai/models) for current availability
|
||||
- Use the `:free` suffix for free model variants
|
||||
- Model IDs are case-sensitive
|
||||
|
||||
### High Token Usage Warning
|
||||
|
||||
If you see warnings about high token usage (>50,000 per request):
|
||||
- Reduce `CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES`
|
||||
- Reduce `CLAUDE_MEM_OPENROUTER_MAX_TOKENS`
|
||||
- Consider a model with larger context window
|
||||
|
||||
### Connection Errors
|
||||
|
||||
If you see connection errors:
|
||||
- Check your internet connection
|
||||
- Verify OpenRouter service status at [status.openrouter.ai](https://status.openrouter.ai)
|
||||
- The agent will automatically fall back to Claude
|
||||
|
||||
## API Details
|
||||
|
||||
OpenRouter uses an OpenAI-compatible REST API:
|
||||
|
||||
**Endpoint**: `https://openrouter.ai/api/v1/chat/completions`
|
||||
|
||||
**Headers**:
|
||||
```
|
||||
Authorization: Bearer {apiKey}
|
||||
HTTP-Referer: https://github.com/thedotmack/claude-mem
|
||||
X-Title: claude-mem
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
**Request Format**:
|
||||
```json
|
||||
{
|
||||
"model": "xiaomi/mimo-v2-flash:free",
|
||||
"messages": [
|
||||
{"role": "system", "content": "..."},
|
||||
{"role": "user", "content": "..."}
|
||||
],
|
||||
"temperature": 0.3,
|
||||
"max_tokens": 4096
|
||||
}
|
||||
```
|
||||
|
||||
## Comparing Providers
|
||||
|
||||
| Feature | Claude (SDK) | Gemini | OpenRouter |
|
||||
|---------|-------------|--------|------------|
|
||||
| **Cost** | Pay per token | Free tier + paid | Free models + paid |
|
||||
| **Models** | Claude only | Gemini only | 100+ models |
|
||||
| **Quality** | Highest | High | Varies by model |
|
||||
| **Rate limits** | Based on tier | 5-4000 RPM | Varies by model |
|
||||
| **Fallback** | N/A (primary) | → Claude | → Claude |
|
||||
| **Setup** | Automatic | API key required | API key required |
|
||||
|
||||
<Tip>
|
||||
**Recommendation**: Start with OpenRouter's free `xiaomi/mimo-v2-flash:free` model for zero-cost observation extraction. If you need higher quality or encounter rate limits, switch to Claude or add OpenRouter credits for premium models.
|
||||
</Tip>
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Configuration](../configuration) - Full settings reference
|
||||
- [Gemini Provider](gemini-provider) - Alternative free provider
|
||||
- [Getting Started](getting-started) - Basic usage guide
|
||||
- [Troubleshooting](../troubleshooting) - Common issues
|
||||
@@ -176,7 +176,7 @@ This design ensures that private content never reaches the database, search indi
|
||||
1. Verify correct syntax: `<private>content</private>`
|
||||
2. Check `~/.claude-mem/silent.log` for errors
|
||||
3. Ensure worker is running: `npm run worker:status`
|
||||
4. Restart worker: `claude-mem restart`
|
||||
4. Restart worker: `npm run worker:restart`
|
||||
|
||||
### Partial Content Stored
|
||||
|
||||
|
||||
@@ -364,7 +364,7 @@ If search isn't working, check the worker service:
|
||||
|
||||
```bash
|
||||
npm run worker:status # Check worker status
|
||||
claude-mem restart # Restart if needed
|
||||
npm run worker:restart # Restart if needed
|
||||
npm run worker:logs # View logs
|
||||
```
|
||||
|
||||
|
||||
+10
-6
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "8.0.2",
|
||||
"version": "8.2.4",
|
||||
"description": "Memory compression system for Claude Code - persist context across sessions",
|
||||
"keywords": [
|
||||
"claude",
|
||||
@@ -31,18 +31,22 @@
|
||||
"bun": ">=1.0.0"
|
||||
},
|
||||
"scripts": {
|
||||
"dev": "npm run build-and-sync",
|
||||
"build": "node scripts/build-hooks.js",
|
||||
"build-and-sync": "npm run build && npm run sync-marketplace && sleep 1 && cd ~/.claude/plugins/marketplaces/thedotmack && npm run worker:restart",
|
||||
"sync-marketplace": "node scripts/sync-marketplace.cjs",
|
||||
"sync-marketplace:force": "node scripts/sync-marketplace.cjs --force",
|
||||
"build:binaries": "node scripts/build-worker-binary.js",
|
||||
"worker:start": "claude-mem start",
|
||||
"worker:stop": "claude-mem stop",
|
||||
"worker:restart": "claude-mem restart",
|
||||
"worker:status": "claude-mem status",
|
||||
"worker:logs": "tail -n 50 ~/.claude-mem/logs/worker-$(date +%Y-%m-%d).log",
|
||||
"worker:tail": "tail -f 50 ~/.claude-mem/logs/worker-$(date +%Y-%m-%d).log",
|
||||
"changelog:generate": "node scripts/generate-changelog.js",
|
||||
"discord:notify": "node scripts/discord-release-notify.js",
|
||||
"worker:start": "bun plugin/scripts/worker-cli.js start",
|
||||
"worker:stop": "bun plugin/scripts/worker-cli.js stop",
|
||||
"worker:restart": "bun plugin/scripts/worker-cli.js restart",
|
||||
"worker:status": "bun plugin/scripts/worker-cli.js status",
|
||||
"queue:check": "bun scripts/check-pending-queue.ts",
|
||||
"queue:process": "bun scripts/check-pending-queue.ts --process",
|
||||
"translate-readme": "bun scripts/translate-readme/cli.ts -v -o docs/i18n README.md",
|
||||
"translate:tier1": "npm run translate-readme -- zh ja pt-br ko es de fr",
|
||||
"translate:tier2": "npm run translate-readme -- he ar ru pl cs nl tr uk",
|
||||
@@ -52,7 +56,7 @@
|
||||
"bug-report": "npx tsx scripts/bug-report/cli.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@anthropic-ai/claude-agent-sdk": "^0.1.67",
|
||||
"@anthropic-ai/claude-agent-sdk": "^0.1.76",
|
||||
"@modelcontextprotocol/sdk": "^1.20.1",
|
||||
"ansi-to-html": "^0.7.2",
|
||||
"express": "^4.18.2",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "8.0.2",
|
||||
"version": "8.2.4",
|
||||
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
|
||||
"author": {
|
||||
"name": "Alex Newman"
|
||||
|
||||
+28
-14
@@ -7,7 +7,17 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/smart-install.js\" && node \"${CLAUDE_PLUGIN_ROOT}/scripts/context-hook.js\"",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/smart-install.js\"",
|
||||
"timeout": 300
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "bun \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 180
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/context-hook.js\"",
|
||||
"timeout": 300
|
||||
},
|
||||
{
|
||||
@@ -21,10 +31,15 @@
|
||||
"UserPromptSubmit": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "bun \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 180
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/new-hook.js\"",
|
||||
"timeout": 120
|
||||
"timeout": 300
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -33,10 +48,15 @@
|
||||
{
|
||||
"matcher": "*",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "bun \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 180
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/save-hook.js\"",
|
||||
"timeout": 120
|
||||
"timeout": 300
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -46,19 +66,13 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js\"",
|
||||
"timeout": 120
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"SessionEnd": [
|
||||
{
|
||||
"hooks": [
|
||||
"command": "bun \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 180
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/cleanup-hook.js\"",
|
||||
"timeout": 120
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js\"",
|
||||
"timeout": 300
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -82,7 +82,7 @@
|
||||
"system_identity": "You are a Claude-Mem, a specialized observer tool for creating searchable memory FOR FUTURE SESSIONS.\n\nCRITICAL: Record what was DISCOVERED/IDENTIFIED/REVEALED about the investigation, not what you (the observer) are doing.\n\nYou do not have access to tools. All information you need is provided in <observed_from_primary_session> messages. Create observations from what you observe - no investigation needed.",
|
||||
"spatial_awareness": "SPATIAL AWARENESS: Tool executions include the working directory (tool_cwd) to help you understand:\n- Which investigation folder/project is being worked on\n- Where email files are located relative to the project root\n- How to match requested paths to actual execution paths",
|
||||
"observer_role": "Your job is to monitor an email fraud investigation happening RIGHT NOW, with the goal of creating observations about entities, relationships, timeline events, and evidence as they are discovered LIVE. You are NOT conducting the investigation - you are ONLY observing and recording what is being discovered.",
|
||||
"recording_focus": "WHAT TO RECORD\n--------------\nFocus on investigative elements:\n- New entities discovered (people, organizations, email addresses)\n- Relationships between entities (who contacted whom, organizational ties)\n- Timeline events (when things happened, communication sequences)\n- Evidence supporting or refuting fraud patterns\n- Anomalies or red flags detected\n\nUse verbs like: identified, discovered, revealed, detected, corroborated, confirmed\n\n✅ GOOD EXAMPLES (describes what was discovered):\n- \"John Smith <john@example.com> sent 15 emails requesting wire transfers\"\n- \"Timeline reveals communication pattern between suspicious accounts\"\n- \"Email headers show spoofed sender domain\"\n\n❌ BAD EXAMPLES (describes observation process - DO NOT DO THIS):\n- \"Analyzed email headers and recorded findings\"\n- \"Tracked communication patterns and logged results\"\n- \"Monitored entity relationships and stored data\"",
|
||||
"recording_focus": "WHAT TO RECORD\n--------------\nFocus on investigative elements:\n- New entities discovered (people, organizations, email addresses)\n- Relationships between entities (who contacted whom, organizational ties)\n- Timeline events (when things happened, communication sequences)\n- Evidence supporting or refuting fraud patterns\n- Anomalies or red flags detected\n\nCRITICAL OBSERVATION GRANULARITY:\n- Break up the information into multiple observations as necessary\n- Create AT LEAST 1 observation per tool use\n- When a single tool use returns rich information (like reading an email), create multiple smaller, focused observations rather than one large observation\n- Each observation should be atomic and semantically focused on ONE investigative element\n- Example: One email might yield 3-5 observations (entity discovery, timeline event, relationship, evidence, anomaly)\n\nUse verbs like: identified, discovered, revealed, detected, corroborated, confirmed\n\n✅ GOOD EXAMPLES (describes what was discovered):\n- \"John Smith <john@example.com> sent 15 emails requesting wire transfers\"\n- \"Timeline reveals communication pattern between suspicious accounts\"\n- \"Email headers show spoofed sender domain\"\n\n❌ BAD EXAMPLES (describes observation process - DO NOT DO THIS):\n- \"Analyzed email headers and recorded findings\"\n- \"Tracked communication patterns and logged results\"\n- \"Monitored entity relationships and stored data\"",
|
||||
"skip_guidance": "WHEN TO SKIP\n------------\nSkip routine operations:\n- Empty searches with no results\n- Simple file listings\n- Repetitive operations you've already documented\n- If email research comes back as empty or not found\n- **No output necessary if skipping.**",
|
||||
"type_guidance": "**type**: MUST be EXACTLY one of these options:\n - entity: new person, organization, or email address identified\n - relationship: connection between entities discovered\n - timeline-event: time-stamped event in communication sequence\n - evidence: supporting documentation or proof discovered\n - anomaly: suspicious pattern or irregularity detected\n - conclusion: investigative finding or determination",
|
||||
"concept_guidance": "**concepts**: 2-5 knowledge-type categories. MUST use ONLY these exact keywords:\n - who: people and organizations involved\n - when: timing and sequence of events\n - what-happened: events and communications\n - motive: intent or purpose behind actions\n - red-flag: warning signs of fraud or deception\n - corroboration: evidence supporting a claim",
|
||||
@@ -110,6 +110,11 @@
|
||||
"header_summary_checkpoint": "INVESTIGATION SUMMARY CHECKPOINT\n================================",
|
||||
|
||||
"continuation_greeting": "Hello memory agent, you are continuing to observe the email fraud investigation session.",
|
||||
"continuation_instruction": "IMPORTANT: Continue generating observations from tool use messages using the XML structure below."
|
||||
"continuation_instruction": "IMPORTANT: Continue generating observations from tool use messages using the XML structure below.",
|
||||
|
||||
"summary_instruction": "Write progress notes of what was discovered, what entities were identified, and what investigation steps are next. This is a checkpoint to capture investigation progress so far. The session is ongoing - you may receive more tool executions after this summary. Write \"next_steps\" as the current trajectory of investigation (what's actively being examined or coming up next), not as post-session future work. Always write at least a minimal summary explaining current investigation progress, even if work is still in early stages.",
|
||||
"summary_context_label": "Claude's Full Investigation Response:",
|
||||
"summary_format_instruction": "Respond in this XML format:",
|
||||
"summary_footer": "IMPORTANT! DO NOT do any work right now other than generating this next INVESTIGATION SUMMARY - and remember that you are a memory agent designed to summarize a DIFFERENT investigation session, not this one.\n\nNever reference yourself or your own actions. Do not output anything other than the summary content formatted in the XML structure above. All other output is ignored by the system.\n\nThank you, this summary will be very useful for tracking investigation progress!"
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem-plugin",
|
||||
"version": "8.0.2",
|
||||
"version": "8.2.4",
|
||||
"private": true,
|
||||
"description": "Runtime dependencies for claude-mem bundled hooks",
|
||||
"type": "module",
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+13
-24
File diff suppressed because one or more lines are too long
+13
-24
File diff suppressed because one or more lines are too long
@@ -291,7 +291,7 @@ function installCLI() {
|
||||
console.error('📋 Add to PATH (run once in PowerShell as Admin):');
|
||||
console.error(` [Environment]::SetEnvironmentVariable("Path", $env:Path + ";${cliDir}", "User")`);
|
||||
console.error('');
|
||||
console.error(' Then restart your terminal and use: claude-mem start|stop|restart|status');
|
||||
console.error(' Then restart your terminal and use: npm run worker:start|stop|restart|status');
|
||||
} catch (error) {
|
||||
console.error(`⚠️ Could not install CLI: ${error.message}`);
|
||||
console.error(` You can still use: bun "${WORKER_CLI}" <command>`);
|
||||
@@ -333,9 +333,9 @@ exec "${bunPath}" "${WORKER_CLI}" "$@"
|
||||
console.error('📋 Add to PATH (add to ~/.bashrc or ~/.zshrc):');
|
||||
console.error(' export PATH="$HOME/.local/bin:$PATH"');
|
||||
console.error('');
|
||||
console.error(' Then restart your terminal and use: claude-mem start|stop|restart|status');
|
||||
console.error(' Then restart your terminal and use: npm run worker:start|stop|restart|status');
|
||||
} else {
|
||||
console.error(' Usage: claude-mem start|stop|restart|status');
|
||||
console.error(' Usage: npm run worker:start|stop|restart|status');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`⚠️ Could not install CLI: ${error.message}`);
|
||||
@@ -439,8 +439,31 @@ try {
|
||||
|
||||
// Step 3: Install dependencies if needed
|
||||
if (needsInstall()) {
|
||||
const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8'));
|
||||
const newVersion = pkg.version;
|
||||
|
||||
installDeps();
|
||||
console.error('✅ Dependencies installed');
|
||||
|
||||
// Auto-restart worker to pick up new code
|
||||
const port = process.env.CLAUDE_MEM_WORKER_PORT || 37777;
|
||||
console.error(`[claude-mem] Plugin updated to v${newVersion} - restarting worker...`);
|
||||
try {
|
||||
// Graceful shutdown via HTTP (curl is cross-platform enough)
|
||||
execSync(`curl -s -X POST http://127.0.0.1:${port}/api/admin/shutdown`, {
|
||||
stdio: 'ignore',
|
||||
shell: IS_WINDOWS,
|
||||
timeout: 5000
|
||||
});
|
||||
// Brief wait for port to free
|
||||
execSync(IS_WINDOWS ? 'timeout /t 1 /nobreak >nul' : 'sleep 0.5', {
|
||||
stdio: 'ignore',
|
||||
shell: true
|
||||
});
|
||||
} catch {
|
||||
// Worker wasn't running or already stopped - that's fine
|
||||
}
|
||||
// Worker will be started fresh by next hook in chain (worker-service.cjs start)
|
||||
}
|
||||
|
||||
// Step 4: Install CLI to PATH
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+292
-345
File diff suppressed because one or more lines are too long
Binary file not shown.
@@ -288,7 +288,7 @@ npm run worker:status
|
||||
If the worker is stopped, restart it:
|
||||
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
```
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ npm run worker:status
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
npm install && \
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Fix: Stale PID File
|
||||
@@ -70,7 +70,7 @@ curl -s http://127.0.0.1:37777/health
|
||||
mkdir -p ~/.claude-mem && \
|
||||
echo '{"CLAUDE_MEM_WORKER_PORT":"37778"}' > ~/.claude-mem/settings.json && \
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
claude-mem restart && \
|
||||
npm run worker:restart && \
|
||||
sleep 2 && \
|
||||
curl -s http://127.0.0.1:37778/health
|
||||
```
|
||||
@@ -86,7 +86,7 @@ curl -s http://127.0.0.1:37778/health
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup && \
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;" && \
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
**If integrity check fails, recreate database:**
|
||||
@@ -94,7 +94,7 @@ claude-mem restart
|
||||
# WARNING: This deletes all memory data
|
||||
mv ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.old && \
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Fix: Clean Reinstall
|
||||
@@ -135,7 +135,7 @@ find ~/.claude-mem/logs/ -name "worker-*.log" -mtime +7 -delete
|
||||
|
||||
# Restart worker for fresh log
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
**Note:** Logs auto-rotate daily, manual cleanup rarely needed.
|
||||
|
||||
@@ -29,7 +29,7 @@ Quick fixes for frequently encountered claude-mem problems.
|
||||
3. Restart worker and start new session:
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
4. Create a test observation: `/skill version-bump` then cancel
|
||||
@@ -173,7 +173,7 @@ Quick fixes for frequently encountered claude-mem problems.
|
||||
4. If FTS5 out of sync, restart worker (triggers reindex):
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Issue: Port Conflicts
|
||||
@@ -194,7 +194,7 @@ Quick fixes for frequently encountered claude-mem problems.
|
||||
mkdir -p ~/.claude-mem
|
||||
echo '{"CLAUDE_MEM_WORKER_PORT":"37778"}' > ~/.claude-mem/settings.json
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Issue: Database Corrupted
|
||||
@@ -219,7 +219,7 @@ Quick fixes for frequently encountered claude-mem problems.
|
||||
```bash
|
||||
rm ~/.claude-mem/claude-mem.db
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
# Worker will create new database
|
||||
```
|
||||
|
||||
|
||||
@@ -173,7 +173,7 @@ If FTS5 counts don't match, triggers may have failed. Restart worker to rebuild:
|
||||
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
The worker will rebuild FTS5 indexes on startup if they're out of sync.
|
||||
@@ -263,7 +263,7 @@ sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
```bash
|
||||
mv ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.archive
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Database Recovery
|
||||
|
||||
@@ -13,7 +13,7 @@ npm run worker:status
|
||||
npm run worker:start
|
||||
|
||||
# Restart worker
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
|
||||
# Stop worker
|
||||
npm run worker:stop
|
||||
|
||||
@@ -152,7 +152,7 @@ npm run worker:start
|
||||
```bash
|
||||
# Restart worker (stops and starts)
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
|
||||
# Or manually stop and start
|
||||
npm run worker:stop
|
||||
@@ -219,7 +219,7 @@ npm run worker:start
|
||||
**Port conflict:**
|
||||
```bash
|
||||
echo '{"CLAUDE_MEM_WORKER_PORT":"37778"}' > ~/.claude-mem/settings.json
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
**Stale PID file:**
|
||||
@@ -261,14 +261,14 @@ If fails, backup and recreate database.
|
||||
**Out of memory:**
|
||||
Check if database is too large or memory leak. Restart:
|
||||
```bash
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
**Port conflict race condition:**
|
||||
Another process grabbing port intermittently. Change port:
|
||||
```bash
|
||||
echo '{"CLAUDE_MEM_WORKER_PORT":"37778"}' > ~/.claude-mem/settings.json
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Worker Management Commands
|
||||
@@ -284,7 +284,7 @@ npm run worker:start
|
||||
npm run worker:stop
|
||||
|
||||
# Restart worker
|
||||
claude-mem restart
|
||||
npm run worker:restart
|
||||
|
||||
# View logs
|
||||
npm run worker:logs
|
||||
@@ -355,7 +355,7 @@ All should return appropriate responses (HTML for viewer, JSON for APIs).
|
||||
|---------|---------|----------------|
|
||||
| Check if running | `npm run worker:status` | Shows PID and uptime |
|
||||
| Worker not running | `npm run worker:start` | Worker starts successfully |
|
||||
| Worker crashed | `claude-mem restart` | Worker restarts |
|
||||
| Worker crashed | `npm run worker:restart` | Worker restarts |
|
||||
| View recent errors | `grep -i error ~/.claude-mem/logs/worker-$(date +%Y-%m-%d).log \| tail -20` | Shows recent errors |
|
||||
| Port in use | `lsof -i :37777` | Shows process using port |
|
||||
| Stale PID | `rm ~/.claude-mem/worker.pid && npm run worker:start` | Removes stale PID and starts |
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1817,6 +1817,49 @@
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
.modal-footer {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
align-items: center;
|
||||
gap: 16px;
|
||||
padding: 16px 24px;
|
||||
border-top: 1px solid var(--modal-border);
|
||||
background: var(--modal-header-bg);
|
||||
}
|
||||
|
||||
.modal-footer .save-status {
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.modal-footer .save-status .success {
|
||||
color: var(--success-color, #22c55e);
|
||||
}
|
||||
|
||||
.modal-footer .save-status .error {
|
||||
color: var(--error-color, #ef4444);
|
||||
}
|
||||
|
||||
.modal-footer .save-btn {
|
||||
padding: 8px 24px;
|
||||
background: var(--accent-color, #3b82f6);
|
||||
color: white;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
font-size: 14px;
|
||||
font-weight: 500;
|
||||
cursor: pointer;
|
||||
transition: background 0.15s ease;
|
||||
}
|
||||
|
||||
.modal-footer .save-btn:hover:not(:disabled) {
|
||||
background: var(--accent-hover, #2563eb);
|
||||
}
|
||||
|
||||
.modal-footer .save-btn:disabled {
|
||||
opacity: 0.6;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
/* Preview Column - Terminal Style */
|
||||
.preview-column {
|
||||
padding: 20px;
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
import { query } from "@anthropic-ai/claude-agent-sdk";
|
||||
import * as fs from "fs";
|
||||
import * as path from "path";
|
||||
|
||||
const pathToFolder = "/Users/alexnewman/Scripts/claude-mem/datasets/epstein-mode/";
|
||||
const pathToPlugin = "/Users/alexnewman/Scripts/claude-mem/plugin/";
|
||||
const WORKER_PORT = 37777;
|
||||
|
||||
// Or read from a directory
|
||||
const filesToProcess = fs
|
||||
.readdirSync(pathToFolder)
|
||||
.filter((f) => f.endsWith(".md"))
|
||||
.sort((a, b) => {
|
||||
// Extract numeric part from filename (e.g., "0001.md" -> 1)
|
||||
const numA = parseInt(a.match(/\d+/)?.[0] || "0", 10);
|
||||
const numB = parseInt(b.match(/\d+/)?.[0] || "0", 10);
|
||||
return numA - numB;
|
||||
})
|
||||
.map((f) => path.join(pathToFolder, f));
|
||||
|
||||
/**
|
||||
* Poll the worker's processing status endpoint until the queue is empty
|
||||
*/
|
||||
async function waitForQueueToEmpty(): Promise<void> {
|
||||
const maxWaitTimeMs = 5 * 60 * 1000; // 5 minutes maximum
|
||||
const pollIntervalMs = 500; // Poll every 500ms
|
||||
const startTime = Date.now();
|
||||
|
||||
while (true) {
|
||||
try {
|
||||
const response = await fetch(`http://localhost:${WORKER_PORT}/api/processing-status`);
|
||||
if (!response.ok) {
|
||||
console.error(`Failed to get processing status: ${response.status}`);
|
||||
break;
|
||||
}
|
||||
|
||||
const status = await response.json();
|
||||
console.log(`Queue status - Processing: ${status.isProcessing}, Queue depth: ${status.queueDepth}`);
|
||||
|
||||
// Exit when queue is empty
|
||||
if (status.queueDepth === 0 && !status.isProcessing) {
|
||||
console.log("Queue is empty, continuing to next prompt");
|
||||
break;
|
||||
}
|
||||
|
||||
// Check timeout
|
||||
if (Date.now() - startTime > maxWaitTimeMs) {
|
||||
console.warn("Warning: Queue did not empty within timeout, continuing anyway");
|
||||
break;
|
||||
}
|
||||
|
||||
// Wait before polling again
|
||||
await new Promise(resolve => setTimeout(resolve, pollIntervalMs));
|
||||
} catch (error) {
|
||||
console.error("Error polling worker status:", error);
|
||||
// On error, wait a bit and continue to avoid infinite loop
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// var i = 0;
|
||||
|
||||
for (const file of filesToProcess) {
|
||||
// i++;
|
||||
// Limit for testing
|
||||
// if (i > 3) break;
|
||||
|
||||
console.log(`\n=== Processing ${file} ===\n`);
|
||||
|
||||
for await (const message of query({
|
||||
prompt: `Read ${file} and think about how it relates to the injected context above (if any).`,
|
||||
options: {
|
||||
cwd: pathToFolder,
|
||||
plugins: [{ type: "local", path: pathToPlugin }],
|
||||
},
|
||||
})) {
|
||||
if (message.type === "system" && message.subtype === "init") {
|
||||
console.log("Plugins:", message.plugins);
|
||||
console.log("Commands:", message.slash_commands);
|
||||
}
|
||||
|
||||
if (message.type === "assistant") {
|
||||
console.log("Assistant:", message.message.content);
|
||||
}
|
||||
console.log("Raw:", JSON.stringify(message, null, 2));
|
||||
}
|
||||
|
||||
// Wait for the worker queue to be empty before continuing to the next file
|
||||
console.log("\n=== Waiting for worker queue to empty ===\n");
|
||||
await waitForQueueToEmpty();
|
||||
}
|
||||
@@ -17,7 +17,6 @@ const HOOKS = [
|
||||
{ name: 'new-hook', source: 'src/hooks/new-hook.ts' },
|
||||
{ name: 'save-hook', source: 'src/hooks/save-hook.ts' },
|
||||
{ name: 'summary-hook', source: 'src/hooks/summary-hook.ts' },
|
||||
{ name: 'cleanup-hook', source: 'src/hooks/cleanup-hook.ts' },
|
||||
{ name: 'user-message-hook', source: 'src/hooks/user-message-hook.ts' }
|
||||
];
|
||||
|
||||
@@ -26,11 +25,6 @@ const WORKER_SERVICE = {
|
||||
source: 'src/services/worker-service.ts'
|
||||
};
|
||||
|
||||
const WORKER_WRAPPER = {
|
||||
name: 'worker-wrapper',
|
||||
source: 'src/services/worker-wrapper.ts'
|
||||
};
|
||||
|
||||
const MCP_SERVER = {
|
||||
name: 'mcp-server',
|
||||
source: 'src/servers/mcp-server.ts'
|
||||
@@ -41,11 +35,6 @@ const CONTEXT_GENERATOR = {
|
||||
source: 'src/services/context-generator.ts'
|
||||
};
|
||||
|
||||
const WORKER_CLI = {
|
||||
name: 'worker-cli',
|
||||
source: 'src/cli/worker-cli.ts'
|
||||
};
|
||||
|
||||
async function buildHooks() {
|
||||
console.log('🔨 Building claude-mem hooks and worker service...\n');
|
||||
|
||||
@@ -125,31 +114,6 @@ async function buildHooks() {
|
||||
const workerStats = fs.statSync(`${hooksDir}/${WORKER_SERVICE.name}.cjs`);
|
||||
console.log(`✓ worker-service built (${(workerStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build worker wrapper (Windows zombie port fix)
|
||||
console.log(`\n🔧 Building worker wrapper...`);
|
||||
await build({
|
||||
entryPoints: [WORKER_WRAPPER.source],
|
||||
bundle: true,
|
||||
platform: 'node',
|
||||
target: 'node18',
|
||||
format: 'cjs',
|
||||
outfile: `${hooksDir}/${WORKER_WRAPPER.name}.cjs`,
|
||||
minify: true,
|
||||
logLevel: 'error',
|
||||
external: ['bun:sqlite'],
|
||||
define: {
|
||||
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
|
||||
},
|
||||
banner: {
|
||||
js: '#!/usr/bin/env bun'
|
||||
}
|
||||
});
|
||||
|
||||
// Make worker wrapper executable
|
||||
fs.chmodSync(`${hooksDir}/${WORKER_WRAPPER.name}.cjs`, 0o755);
|
||||
const wrapperStats = fs.statSync(`${hooksDir}/${WORKER_WRAPPER.name}.cjs`);
|
||||
console.log(`✓ worker-wrapper built (${(wrapperStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build MCP server
|
||||
console.log(`\n🔧 Building MCP server...`);
|
||||
await build({
|
||||
@@ -195,31 +159,6 @@ async function buildHooks() {
|
||||
const contextGenStats = fs.statSync(`${hooksDir}/${CONTEXT_GENERATOR.name}.cjs`);
|
||||
console.log(`✓ context-generator built (${(contextGenStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build worker CLI
|
||||
console.log(`\n🔧 Building worker CLI...`);
|
||||
await build({
|
||||
entryPoints: [WORKER_CLI.source],
|
||||
bundle: true,
|
||||
platform: 'node',
|
||||
target: 'node18',
|
||||
format: 'esm',
|
||||
outfile: `${hooksDir}/${WORKER_CLI.name}.js`,
|
||||
minify: true,
|
||||
logLevel: 'error',
|
||||
external: ['bun:sqlite'],
|
||||
define: {
|
||||
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
|
||||
},
|
||||
banner: {
|
||||
js: '#!/usr/bin/env bun'
|
||||
}
|
||||
});
|
||||
|
||||
// Make worker CLI executable
|
||||
fs.chmodSync(`${hooksDir}/${WORKER_CLI.name}.js`, 0o755);
|
||||
const workerCliStats = fs.statSync(`${hooksDir}/${WORKER_CLI.name}.js`);
|
||||
console.log(`✓ worker-cli built (${(workerCliStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build each hook
|
||||
for (const hook of HOOKS) {
|
||||
console.log(`\n🔧 Building ${hook.name}...`);
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* Check and process pending observation queue
|
||||
*
|
||||
* Usage:
|
||||
* bun scripts/check-pending-queue.ts # Check status and prompt to process
|
||||
* bun scripts/check-pending-queue.ts --process # Auto-process without prompting
|
||||
* bun scripts/check-pending-queue.ts --limit 5 # Process up to 5 sessions
|
||||
*/
|
||||
|
||||
const WORKER_URL = 'http://localhost:37777';
|
||||
|
||||
interface QueueMessage {
|
||||
id: number;
|
||||
session_db_id: number;
|
||||
message_type: string;
|
||||
tool_name: string | null;
|
||||
status: 'pending' | 'processing' | 'failed';
|
||||
retry_count: number;
|
||||
created_at_epoch: number;
|
||||
project: string | null;
|
||||
}
|
||||
|
||||
interface QueueResponse {
|
||||
queue: {
|
||||
messages: QueueMessage[];
|
||||
totalPending: number;
|
||||
totalProcessing: number;
|
||||
totalFailed: number;
|
||||
stuckCount: number;
|
||||
};
|
||||
recentlyProcessed: QueueMessage[];
|
||||
sessionsWithPendingWork: number[];
|
||||
}
|
||||
|
||||
interface ProcessResponse {
|
||||
success: boolean;
|
||||
totalPendingSessions: number;
|
||||
sessionsStarted: number;
|
||||
sessionsSkipped: number;
|
||||
startedSessionIds: number[];
|
||||
}
|
||||
|
||||
async function checkWorkerHealth(): Promise<boolean> {
|
||||
try {
|
||||
const res = await fetch(`${WORKER_URL}/api/health`);
|
||||
return res.ok;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function getQueueStatus(): Promise<QueueResponse> {
|
||||
const res = await fetch(`${WORKER_URL}/api/pending-queue`);
|
||||
if (!res.ok) {
|
||||
throw new Error(`Failed to get queue status: ${res.status}`);
|
||||
}
|
||||
return res.json();
|
||||
}
|
||||
|
||||
async function processQueue(limit: number): Promise<ProcessResponse> {
|
||||
const res = await fetch(`${WORKER_URL}/api/pending-queue/process`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ sessionLimit: limit })
|
||||
});
|
||||
if (!res.ok) {
|
||||
throw new Error(`Failed to process queue: ${res.status}`);
|
||||
}
|
||||
return res.json();
|
||||
}
|
||||
|
||||
function formatAge(epochMs: number): string {
|
||||
const ageMs = Date.now() - epochMs;
|
||||
const minutes = Math.floor(ageMs / 60000);
|
||||
const hours = Math.floor(minutes / 60);
|
||||
const days = Math.floor(hours / 24);
|
||||
|
||||
if (days > 0) return `${days}d ${hours % 24}h ago`;
|
||||
if (hours > 0) return `${hours}h ${minutes % 60}m ago`;
|
||||
return `${minutes}m ago`;
|
||||
}
|
||||
|
||||
async function prompt(question: string): Promise<string> {
|
||||
// Check if we have a TTY for interactive input
|
||||
if (!process.stdin.isTTY) {
|
||||
console.log(question + '(no TTY, use --process flag for non-interactive mode)');
|
||||
return 'n';
|
||||
}
|
||||
|
||||
return new Promise((resolve) => {
|
||||
process.stdout.write(question);
|
||||
process.stdin.setRawMode(false);
|
||||
process.stdin.resume();
|
||||
process.stdin.once('data', (data) => {
|
||||
process.stdin.pause();
|
||||
resolve(data.toString().trim());
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
|
||||
// Help flag
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
console.log(`
|
||||
Claude-Mem Pending Queue Manager
|
||||
|
||||
Check and process pending observation queue backlog.
|
||||
|
||||
Usage:
|
||||
bun scripts/check-pending-queue.ts [options]
|
||||
|
||||
Options:
|
||||
--help, -h Show this help message
|
||||
--process Auto-process without prompting
|
||||
--limit N Process up to N sessions (default: 10)
|
||||
|
||||
Examples:
|
||||
# Check queue status interactively
|
||||
bun scripts/check-pending-queue.ts
|
||||
|
||||
# Auto-process up to 10 sessions
|
||||
bun scripts/check-pending-queue.ts --process
|
||||
|
||||
# Process up to 5 sessions
|
||||
bun scripts/check-pending-queue.ts --process --limit 5
|
||||
|
||||
What is this for?
|
||||
If the claude-mem worker crashes or restarts, pending observations may
|
||||
be left unprocessed. This script shows the backlog and lets you trigger
|
||||
processing. The worker no longer auto-recovers on startup to give you
|
||||
control over when processing happens.
|
||||
`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const autoProcess = args.includes('--process');
|
||||
const limitArg = args.find((_, i) => args[i - 1] === '--limit');
|
||||
const limit = limitArg ? parseInt(limitArg, 10) : 10;
|
||||
|
||||
console.log('\n=== Claude-Mem Pending Queue Status ===\n');
|
||||
|
||||
// Check worker health
|
||||
const healthy = await checkWorkerHealth();
|
||||
if (!healthy) {
|
||||
console.log('Worker is not running. Start it with:');
|
||||
console.log(' cd ~/.claude/plugins/marketplaces/thedotmack && npm run worker:start\n');
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('Worker status: Running\n');
|
||||
|
||||
// Get queue status
|
||||
const status = await getQueueStatus();
|
||||
const { queue, sessionsWithPendingWork } = status;
|
||||
|
||||
// Display summary
|
||||
console.log('Queue Summary:');
|
||||
console.log(` Pending: ${queue.totalPending}`);
|
||||
console.log(` Processing: ${queue.totalProcessing}`);
|
||||
console.log(` Failed: ${queue.totalFailed}`);
|
||||
console.log(` Stuck: ${queue.stuckCount} (processing > 5 min)`);
|
||||
console.log(` Sessions: ${sessionsWithPendingWork.length} with pending work\n`);
|
||||
|
||||
// Check if there's any backlog
|
||||
const hasBacklog = queue.totalPending > 0 || queue.totalFailed > 0;
|
||||
const hasStuck = queue.stuckCount > 0;
|
||||
|
||||
if (!hasBacklog && !hasStuck) {
|
||||
console.log('No backlog detected. Queue is healthy.\n');
|
||||
|
||||
// Show recently processed if any
|
||||
if (status.recentlyProcessed.length > 0) {
|
||||
console.log(`Recently processed: ${status.recentlyProcessed.length} messages in last 30 min\n`);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Show details about pending messages
|
||||
if (queue.messages.length > 0) {
|
||||
console.log('Pending Messages:');
|
||||
console.log('─'.repeat(80));
|
||||
|
||||
// Group by session
|
||||
const bySession = new Map<number, QueueMessage[]>();
|
||||
for (const msg of queue.messages) {
|
||||
const list = bySession.get(msg.session_db_id) || [];
|
||||
list.push(msg);
|
||||
bySession.set(msg.session_db_id, list);
|
||||
}
|
||||
|
||||
for (const [sessionId, messages] of bySession) {
|
||||
const project = messages[0].project || 'unknown';
|
||||
const oldest = Math.min(...messages.map(m => m.created_at_epoch));
|
||||
const statuses = {
|
||||
pending: messages.filter(m => m.status === 'pending').length,
|
||||
processing: messages.filter(m => m.status === 'processing').length,
|
||||
failed: messages.filter(m => m.status === 'failed').length
|
||||
};
|
||||
|
||||
console.log(` Session ${sessionId} (${project})`);
|
||||
console.log(` Messages: ${messages.length} total`);
|
||||
console.log(` Status: ${statuses.pending} pending, ${statuses.processing} processing, ${statuses.failed} failed`);
|
||||
console.log(` Age: ${formatAge(oldest)}`);
|
||||
}
|
||||
console.log('─'.repeat(80));
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Offer to process
|
||||
if (autoProcess) {
|
||||
console.log(`Auto-processing up to ${limit} sessions...\n`);
|
||||
} else {
|
||||
const answer = await prompt(`Process pending queue? (up to ${limit} sessions) [y/N]: `);
|
||||
if (answer.toLowerCase() !== 'y') {
|
||||
console.log('\nSkipped. Run with --process to auto-process.\n');
|
||||
process.exit(0);
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Process the queue
|
||||
const result = await processQueue(limit);
|
||||
|
||||
console.log('Processing Result:');
|
||||
console.log(` Sessions started: ${result.sessionsStarted}`);
|
||||
console.log(` Sessions skipped: ${result.sessionsSkipped} (already active)`);
|
||||
console.log(` Remaining: ${result.totalPendingSessions - result.sessionsStarted}`);
|
||||
|
||||
if (result.startedSessionIds.length > 0) {
|
||||
console.log(` Started IDs: ${result.startedSessionIds.join(', ')}`);
|
||||
}
|
||||
|
||||
console.log('\nProcessing started in background. Check status again in a few minutes.\n');
|
||||
}
|
||||
|
||||
main().catch(err => {
|
||||
console.error('Error:', err.message);
|
||||
process.exit(1);
|
||||
});
|
||||
Executable
+174
@@ -0,0 +1,174 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Fix ALL Corrupted Observation Timestamps
|
||||
*
|
||||
* This script finds and repairs ALL observations with timestamps that don't match
|
||||
* their session start times, not just ones in an arbitrary "bad window".
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
||||
|
||||
interface CorruptedObservation {
|
||||
obs_id: number;
|
||||
obs_title: string;
|
||||
obs_created: number;
|
||||
session_started: number;
|
||||
session_completed: number | null;
|
||||
sdk_session_id: string;
|
||||
}
|
||||
|
||||
function formatTimestamp(epoch: number): string {
|
||||
return new Date(epoch).toLocaleString('en-US', {
|
||||
timeZone: 'America/Los_Angeles',
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
second: '2-digit'
|
||||
});
|
||||
}
|
||||
|
||||
function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const dryRun = args.includes('--dry-run');
|
||||
const autoYes = args.includes('--yes') || args.includes('-y');
|
||||
|
||||
console.log('🔍 Finding ALL observations with timestamp corruption...\n');
|
||||
if (dryRun) {
|
||||
console.log('🏃 DRY RUN MODE - No changes will be made\n');
|
||||
}
|
||||
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Find all observations where timestamp doesn't match session
|
||||
const corrupted = db.query<CorruptedObservation, []>(`
|
||||
SELECT
|
||||
o.id as obs_id,
|
||||
o.title as obs_title,
|
||||
o.created_at_epoch as obs_created,
|
||||
s.started_at_epoch as session_started,
|
||||
s.completed_at_epoch as session_completed,
|
||||
s.sdk_session_id
|
||||
FROM observations o
|
||||
JOIN sdk_sessions s ON o.sdk_session_id = s.sdk_session_id
|
||||
WHERE o.created_at_epoch < s.started_at_epoch -- Observation older than session
|
||||
OR (s.completed_at_epoch IS NOT NULL
|
||||
AND o.created_at_epoch > (s.completed_at_epoch + 3600000)) -- More than 1hr after session
|
||||
ORDER BY o.id
|
||||
`).all();
|
||||
|
||||
console.log(`Found ${corrupted.length} observations with corrupted timestamps\n`);
|
||||
|
||||
if (corrupted.length === 0) {
|
||||
console.log('✅ No corrupted timestamps found!');
|
||||
db.close();
|
||||
return;
|
||||
}
|
||||
|
||||
// Display findings
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('PROPOSED FIXES:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════\n');
|
||||
|
||||
for (const obs of corrupted.slice(0, 50)) {
|
||||
const daysDiff = Math.round((obs.obs_created - obs.session_started) / (1000 * 60 * 60 * 24));
|
||||
console.log(`Observation #${obs.obs_id}: ${obs.obs_title || '(no title)'}`);
|
||||
console.log(` ❌ Wrong: ${formatTimestamp(obs.obs_created)}`);
|
||||
console.log(` ✅ Correct: ${formatTimestamp(obs.session_started)}`);
|
||||
console.log(` 📅 Off by ${daysDiff} days\n`);
|
||||
}
|
||||
|
||||
if (corrupted.length > 50) {
|
||||
console.log(`... and ${corrupted.length - 50} more\n`);
|
||||
}
|
||||
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log(`Ready to fix ${corrupted.length} observations.`);
|
||||
|
||||
if (dryRun) {
|
||||
console.log('\n🏃 DRY RUN COMPLETE - No changes made.');
|
||||
console.log('Run without --dry-run flag to apply fixes.\n');
|
||||
db.close();
|
||||
return;
|
||||
}
|
||||
|
||||
if (autoYes) {
|
||||
console.log('Auto-confirming with --yes flag...\n');
|
||||
applyFixes(db, corrupted);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('Apply these fixes? (y/n): ');
|
||||
|
||||
const stdin = Bun.stdin.stream();
|
||||
const reader = stdin.getReader();
|
||||
|
||||
reader.read().then(({ value }) => {
|
||||
const response = new TextDecoder().decode(value).trim().toLowerCase();
|
||||
|
||||
if (response === 'y' || response === 'yes') {
|
||||
applyFixes(db, corrupted);
|
||||
} else {
|
||||
console.log('\n❌ Fixes cancelled. No changes made.');
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Error:', error);
|
||||
db.close();
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
function applyFixes(db: Database, corrupted: CorruptedObservation[]) {
|
||||
console.log('\n🔧 Applying fixes...\n');
|
||||
|
||||
const updateStmt = db.prepare(`
|
||||
UPDATE observations
|
||||
SET created_at_epoch = ?,
|
||||
created_at = datetime(?/1000, 'unixepoch')
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
let successCount = 0;
|
||||
let errorCount = 0;
|
||||
|
||||
for (const obs of corrupted) {
|
||||
try {
|
||||
updateStmt.run(
|
||||
obs.session_started,
|
||||
obs.session_started,
|
||||
obs.obs_id
|
||||
);
|
||||
successCount++;
|
||||
if (successCount % 10 === 0 || successCount <= 10) {
|
||||
console.log(`✅ Fixed observation #${obs.obs_id}`);
|
||||
}
|
||||
} catch (error) {
|
||||
errorCount++;
|
||||
console.error(`❌ Failed to fix observation #${obs.obs_id}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('RESULTS:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log(`✅ Successfully fixed: ${successCount}`);
|
||||
console.log(`❌ Failed: ${errorCount}`);
|
||||
console.log(`📊 Total processed: ${corrupted.length}\n`);
|
||||
|
||||
if (successCount > 0) {
|
||||
console.log('🎉 ALL timestamp corruption has been repaired!\n');
|
||||
}
|
||||
|
||||
db.close();
|
||||
}
|
||||
|
||||
main();
|
||||
Executable
+243
@@ -0,0 +1,243 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Fix Corrupted Observation Timestamps
|
||||
*
|
||||
* This script repairs observations that were created during the orphan queue processing
|
||||
* on Dec 24, 2025 between 19:45-20:31. These observations got Dec 24 timestamps instead
|
||||
* of their original timestamps from Dec 17-20.
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
||||
|
||||
// Bad window: Dec 24 19:45-20:31 (timestamps in milliseconds, not microseconds)
|
||||
// Using actual observation epoch format (microseconds since epoch)
|
||||
const BAD_WINDOW_START = 1766623500000; // Dec 24 19:45 PST
|
||||
const BAD_WINDOW_END = 1766626260000; // Dec 24 20:31 PST
|
||||
|
||||
interface AffectedObservation {
|
||||
id: number;
|
||||
sdk_session_id: string;
|
||||
created_at_epoch: number;
|
||||
title: string;
|
||||
}
|
||||
|
||||
interface ProcessedMessage {
|
||||
id: number;
|
||||
session_db_id: number;
|
||||
tool_name: string;
|
||||
created_at_epoch: number;
|
||||
completed_at_epoch: number;
|
||||
}
|
||||
|
||||
interface SessionMapping {
|
||||
session_db_id: number;
|
||||
sdk_session_id: string;
|
||||
}
|
||||
|
||||
interface TimestampFix {
|
||||
observation_id: number;
|
||||
observation_title: string;
|
||||
wrong_timestamp: number;
|
||||
correct_timestamp: number;
|
||||
session_db_id: number;
|
||||
pending_message_id: number;
|
||||
}
|
||||
|
||||
function formatTimestamp(epoch: number): string {
|
||||
return new Date(epoch).toLocaleString('en-US', {
|
||||
timeZone: 'America/Los_Angeles',
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
second: '2-digit'
|
||||
});
|
||||
}
|
||||
|
||||
function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const dryRun = args.includes('--dry-run');
|
||||
const autoYes = args.includes('--yes') || args.includes('-y');
|
||||
|
||||
console.log('🔍 Analyzing corrupted observation timestamps...\n');
|
||||
if (dryRun) {
|
||||
console.log('🏃 DRY RUN MODE - No changes will be made\n');
|
||||
}
|
||||
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Step 1: Find affected observations
|
||||
console.log('Step 1: Finding observations created during bad window...');
|
||||
const affectedObs = db.query<AffectedObservation, []>(`
|
||||
SELECT id, sdk_session_id, created_at_epoch, title
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ${BAD_WINDOW_START}
|
||||
AND created_at_epoch <= ${BAD_WINDOW_END}
|
||||
ORDER BY id
|
||||
`).all();
|
||||
|
||||
console.log(`Found ${affectedObs.length} observations in bad window\n`);
|
||||
|
||||
if (affectedObs.length === 0) {
|
||||
console.log('✅ No affected observations found!');
|
||||
return;
|
||||
}
|
||||
|
||||
// Step 2: Find processed pending_messages from bad window
|
||||
console.log('Step 2: Finding pending messages processed during bad window...');
|
||||
const processedMessages = db.query<ProcessedMessage, []>(`
|
||||
SELECT id, session_db_id, tool_name, created_at_epoch, completed_at_epoch
|
||||
FROM pending_messages
|
||||
WHERE status = 'processed'
|
||||
AND completed_at_epoch >= ${BAD_WINDOW_START}
|
||||
AND completed_at_epoch <= ${BAD_WINDOW_END}
|
||||
ORDER BY completed_at_epoch
|
||||
`).all();
|
||||
|
||||
console.log(`Found ${processedMessages.length} processed messages\n`);
|
||||
|
||||
// Step 3: Match observations to their session start times (simpler approach)
|
||||
console.log('Step 3: Matching observations to session start times...');
|
||||
const fixes: TimestampFix[] = [];
|
||||
|
||||
interface ObsWithSession {
|
||||
obs_id: number;
|
||||
obs_title: string;
|
||||
obs_created: number;
|
||||
session_started: number;
|
||||
sdk_session_id: string;
|
||||
}
|
||||
|
||||
const obsWithSessions = db.query<ObsWithSession, []>(`
|
||||
SELECT
|
||||
o.id as obs_id,
|
||||
o.title as obs_title,
|
||||
o.created_at_epoch as obs_created,
|
||||
s.started_at_epoch as session_started,
|
||||
s.sdk_session_id
|
||||
FROM observations o
|
||||
JOIN sdk_sessions s ON o.sdk_session_id = s.sdk_session_id
|
||||
WHERE o.created_at_epoch >= ${BAD_WINDOW_START}
|
||||
AND o.created_at_epoch <= ${BAD_WINDOW_END}
|
||||
AND s.started_at_epoch < ${BAD_WINDOW_START}
|
||||
ORDER BY o.id
|
||||
`).all();
|
||||
|
||||
for (const row of obsWithSessions) {
|
||||
fixes.push({
|
||||
observation_id: row.obs_id,
|
||||
observation_title: row.obs_title || '(no title)',
|
||||
wrong_timestamp: row.obs_created,
|
||||
correct_timestamp: row.session_started,
|
||||
session_db_id: 0, // Not needed for this approach
|
||||
pending_message_id: 0 // Not needed for this approach
|
||||
});
|
||||
}
|
||||
|
||||
console.log(`Identified ${fixes.length} observations to fix\n`);
|
||||
|
||||
// Step 5: Display what will be fixed
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('PROPOSED FIXES:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════\n');
|
||||
|
||||
for (const fix of fixes) {
|
||||
const daysDiff = Math.round((fix.wrong_timestamp - fix.correct_timestamp) / (1000 * 60 * 60 * 24));
|
||||
console.log(`Observation #${fix.observation_id}: ${fix.observation_title}`);
|
||||
console.log(` ❌ Wrong: ${formatTimestamp(fix.wrong_timestamp)}`);
|
||||
console.log(` ✅ Correct: ${formatTimestamp(fix.correct_timestamp)}`);
|
||||
console.log(` 📅 Off by ${daysDiff} days\n`);
|
||||
}
|
||||
|
||||
// Step 6: Ask for confirmation
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log(`Ready to fix ${fixes.length} observations.`);
|
||||
|
||||
if (dryRun) {
|
||||
console.log('\n🏃 DRY RUN COMPLETE - No changes made.');
|
||||
console.log('Run without --dry-run flag to apply fixes.\n');
|
||||
db.close();
|
||||
return;
|
||||
}
|
||||
|
||||
if (autoYes) {
|
||||
console.log('Auto-confirming with --yes flag...\n');
|
||||
applyFixes(db, fixes);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('Apply these fixes? (y/n): ');
|
||||
|
||||
const stdin = Bun.stdin.stream();
|
||||
const reader = stdin.getReader();
|
||||
|
||||
reader.read().then(({ value }) => {
|
||||
const response = new TextDecoder().decode(value).trim().toLowerCase();
|
||||
|
||||
if (response === 'y' || response === 'yes') {
|
||||
applyFixes(db, fixes);
|
||||
} else {
|
||||
console.log('\n❌ Fixes cancelled. No changes made.');
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Error:', error);
|
||||
db.close();
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
function applyFixes(db: Database, fixes: TimestampFix[]) {
|
||||
console.log('\n🔧 Applying fixes...\n');
|
||||
|
||||
const updateStmt = db.prepare(`
|
||||
UPDATE observations
|
||||
SET created_at_epoch = ?,
|
||||
created_at = datetime(?/1000, 'unixepoch')
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
let successCount = 0;
|
||||
let errorCount = 0;
|
||||
|
||||
for (const fix of fixes) {
|
||||
try {
|
||||
updateStmt.run(
|
||||
fix.correct_timestamp,
|
||||
fix.correct_timestamp,
|
||||
fix.observation_id
|
||||
);
|
||||
successCount++;
|
||||
console.log(`✅ Fixed observation #${fix.observation_id}`);
|
||||
} catch (error) {
|
||||
errorCount++;
|
||||
console.error(`❌ Failed to fix observation #${fix.observation_id}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('RESULTS:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log(`✅ Successfully fixed: ${successCount}`);
|
||||
console.log(`❌ Failed: ${errorCount}`);
|
||||
console.log(`📊 Total processed: ${fixes.length}\n`);
|
||||
|
||||
if (successCount > 0) {
|
||||
console.log('🎉 Timestamp corruption has been repaired!');
|
||||
console.log('💡 Next steps:');
|
||||
console.log(' 1. Verify the fixes with: bun scripts/verify-timestamp-fix.ts');
|
||||
console.log(' 2. Consider re-enabling orphan processing if timestamp fix is working\n');
|
||||
}
|
||||
|
||||
db.close();
|
||||
}
|
||||
|
||||
main();
|
||||
Executable
+143
@@ -0,0 +1,143 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Investigate Timestamp Situation
|
||||
*
|
||||
* This script investigates the actual state of observations and pending messages
|
||||
* to understand what happened with the timestamp corruption.
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
||||
|
||||
function formatTimestamp(epoch: number): string {
|
||||
return new Date(epoch).toLocaleString('en-US', {
|
||||
timeZone: 'America/Los_Angeles',
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
second: '2-digit'
|
||||
});
|
||||
}
|
||||
|
||||
function main() {
|
||||
console.log('🔍 Investigating timestamp situation...\n');
|
||||
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Check 1: Recent observations on Dec 24
|
||||
console.log('Check 1: All observations created on Dec 24, 2025...');
|
||||
const dec24Start = 1735027200000; // Dec 24 00:00 PST
|
||||
const dec24End = 1735113600000; // Dec 25 00:00 PST
|
||||
|
||||
const dec24Obs = db.query(`
|
||||
SELECT id, sdk_session_id, created_at_epoch, title
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ${dec24Start}
|
||||
AND created_at_epoch < ${dec24End}
|
||||
ORDER BY created_at_epoch
|
||||
LIMIT 100
|
||||
`).all();
|
||||
|
||||
console.log(`Found ${dec24Obs.length} observations on Dec 24:\n`);
|
||||
for (const obs of dec24Obs.slice(0, 20)) {
|
||||
console.log(` #${obs.id}: ${formatTimestamp(obs.created_at_epoch)} - ${obs.title || '(no title)'}`);
|
||||
}
|
||||
if (dec24Obs.length > 20) {
|
||||
console.log(` ... and ${dec24Obs.length - 20} more`);
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Check 2: Observations from Dec 17-20
|
||||
console.log('Check 2: Observations from Dec 17-20, 2025...');
|
||||
const dec17Start = 1734422400000; // Dec 17 00:00 PST
|
||||
const dec21Start = 1734768000000; // Dec 21 00:00 PST
|
||||
|
||||
const oldObs = db.query(`
|
||||
SELECT id, sdk_session_id, created_at_epoch, title
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ${dec17Start}
|
||||
AND created_at_epoch < ${dec21Start}
|
||||
ORDER BY created_at_epoch
|
||||
LIMIT 100
|
||||
`).all();
|
||||
|
||||
console.log(`Found ${oldObs.length} observations from Dec 17-20:\n`);
|
||||
for (const obs of oldObs.slice(0, 20)) {
|
||||
console.log(` #${obs.id}: ${formatTimestamp(obs.created_at_epoch)} - ${obs.title || '(no title)'}`);
|
||||
}
|
||||
if (oldObs.length > 20) {
|
||||
console.log(` ... and ${oldObs.length - 20} more`);
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Check 3: Pending messages status
|
||||
console.log('Check 3: Pending messages status...');
|
||||
const statusCounts = db.query(`
|
||||
SELECT status, COUNT(*) as count
|
||||
FROM pending_messages
|
||||
GROUP BY status
|
||||
`).all();
|
||||
|
||||
console.log('Pending message counts by status:');
|
||||
for (const row of statusCounts) {
|
||||
console.log(` ${row.status}: ${row.count}`);
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Check 4: Old pending messages from Dec 17-20
|
||||
console.log('Check 4: Pending messages from Dec 17-20...');
|
||||
const oldMessages = db.query(`
|
||||
SELECT id, session_db_id, tool_name, status, created_at_epoch, completed_at_epoch
|
||||
FROM pending_messages
|
||||
WHERE created_at_epoch >= ${dec17Start}
|
||||
AND created_at_epoch < ${dec21Start}
|
||||
ORDER BY created_at_epoch
|
||||
LIMIT 50
|
||||
`).all();
|
||||
|
||||
console.log(`Found ${oldMessages.length} pending messages from Dec 17-20:\n`);
|
||||
for (const msg of oldMessages.slice(0, 20)) {
|
||||
const completedAt = msg.completed_at_epoch ? formatTimestamp(msg.completed_at_epoch) : 'N/A';
|
||||
console.log(` #${msg.id}: ${msg.tool_name} - Status: ${msg.status}`);
|
||||
console.log(` Created: ${formatTimestamp(msg.created_at_epoch)}`);
|
||||
console.log(` Completed: ${completedAt}\n`);
|
||||
}
|
||||
if (oldMessages.length > 20) {
|
||||
console.log(` ... and ${oldMessages.length - 20} more`);
|
||||
}
|
||||
|
||||
// Check 5: Recently completed pending messages
|
||||
console.log('Check 5: Recently completed pending messages...');
|
||||
const recentCompleted = db.query(`
|
||||
SELECT id, session_db_id, tool_name, status, created_at_epoch, completed_at_epoch
|
||||
FROM pending_messages
|
||||
WHERE completed_at_epoch IS NOT NULL
|
||||
ORDER BY completed_at_epoch DESC
|
||||
LIMIT 20
|
||||
`).all();
|
||||
|
||||
console.log(`Most recent completed pending messages:\n`);
|
||||
for (const msg of recentCompleted) {
|
||||
const createdAt = formatTimestamp(msg.created_at_epoch);
|
||||
const completedAt = formatTimestamp(msg.completed_at_epoch);
|
||||
const lag = Math.round((msg.completed_at_epoch - msg.created_at_epoch) / 1000);
|
||||
console.log(` #${msg.id}: ${msg.tool_name} (${msg.status})`);
|
||||
console.log(` Created: ${createdAt}`);
|
||||
console.log(` Completed: ${completedAt} (${lag}s later)\n`);
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Error:', error);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
main();
|
||||
Executable
+150
@@ -0,0 +1,150 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Validate Timestamp Logic
|
||||
*
|
||||
* This script validates that the backlog timestamp logic would work correctly
|
||||
* by checking pending messages and simulating what timestamps they would get.
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
||||
|
||||
function formatTimestamp(epoch: number): string {
|
||||
return new Date(epoch).toLocaleString('en-US', {
|
||||
timeZone: 'America/Los_Angeles',
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
second: '2-digit'
|
||||
});
|
||||
}
|
||||
|
||||
function main() {
|
||||
console.log('🔍 Validating timestamp logic for backlog processing...\n');
|
||||
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Check for pending messages
|
||||
const pendingStats = db.query(`
|
||||
SELECT
|
||||
status,
|
||||
COUNT(*) as count,
|
||||
MIN(created_at_epoch) as earliest,
|
||||
MAX(created_at_epoch) as latest
|
||||
FROM pending_messages
|
||||
GROUP BY status
|
||||
ORDER BY status
|
||||
`).all();
|
||||
|
||||
console.log('Pending Messages Status:\n');
|
||||
for (const stat of pendingStats) {
|
||||
console.log(`${stat.status}: ${stat.count} messages`);
|
||||
if (stat.earliest && stat.latest) {
|
||||
console.log(` Created: ${formatTimestamp(stat.earliest)} to ${formatTimestamp(stat.latest)}`);
|
||||
}
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Get sample pending messages with their session info
|
||||
const pendingWithSessions = db.query(`
|
||||
SELECT
|
||||
pm.id,
|
||||
pm.session_db_id,
|
||||
pm.tool_name,
|
||||
pm.created_at_epoch as msg_created,
|
||||
pm.status,
|
||||
s.sdk_session_id,
|
||||
s.started_at_epoch as session_started,
|
||||
s.project
|
||||
FROM pending_messages pm
|
||||
LEFT JOIN sdk_sessions s ON pm.session_db_id = s.id
|
||||
WHERE pm.status IN ('pending', 'processing')
|
||||
ORDER BY pm.created_at_epoch
|
||||
LIMIT 10
|
||||
`).all();
|
||||
|
||||
if (pendingWithSessions.length === 0) {
|
||||
console.log('✅ No pending messages - all caught up!\n');
|
||||
db.close();
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`Sample of ${pendingWithSessions.length} pending messages:\n`);
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
|
||||
for (const msg of pendingWithSessions) {
|
||||
console.log(`\nPending Message #${msg.id}: ${msg.tool_name} (${msg.status})`);
|
||||
console.log(` Created: ${formatTimestamp(msg.msg_created)}`);
|
||||
|
||||
if (msg.session_started) {
|
||||
console.log(` Session started: ${formatTimestamp(msg.session_started)}`);
|
||||
console.log(` Project: ${msg.project}`);
|
||||
|
||||
// Validate logic
|
||||
const ageDays = Math.round((Date.now() - msg.msg_created) / (1000 * 60 * 60 * 24));
|
||||
|
||||
if (msg.msg_created < msg.session_started) {
|
||||
console.log(` ⚠️ WARNING: Message created BEFORE session! This is impossible.`);
|
||||
} else if (ageDays > 0) {
|
||||
console.log(` 📅 Message is ${ageDays} days old`);
|
||||
console.log(` ✅ Would use original timestamp: ${formatTimestamp(msg.msg_created)}`);
|
||||
} else {
|
||||
console.log(` ✅ Recent message, would use original timestamp: ${formatTimestamp(msg.msg_created)}`);
|
||||
}
|
||||
} else {
|
||||
console.log(` ⚠️ No session found for session_db_id ${msg.session_db_id}`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('\nTimestamp Logic Validation:\n');
|
||||
console.log('✅ Code Flow:');
|
||||
console.log(' 1. SessionManager.yieldNextMessage() tracks earliestPendingTimestamp');
|
||||
console.log(' 2. SDKAgent captures originalTimestamp before processing');
|
||||
console.log(' 3. processSDKResponse passes originalTimestamp to storeObservation/storeSummary');
|
||||
console.log(' 4. SessionStore uses overrideTimestampEpoch ?? Date.now()');
|
||||
console.log(' 5. earliestPendingTimestamp reset after batch completes\n');
|
||||
|
||||
console.log('✅ Expected Behavior:');
|
||||
console.log(' - New messages: get current timestamp');
|
||||
console.log(' - Backlog messages: get original created_at_epoch');
|
||||
console.log(' - Observations match their source message timestamps\n');
|
||||
|
||||
// Check for any sessions with stuck processing messages
|
||||
const stuckMessages = db.query(`
|
||||
SELECT
|
||||
session_db_id,
|
||||
COUNT(*) as count,
|
||||
MIN(created_at_epoch) as earliest,
|
||||
MAX(created_at_epoch) as latest
|
||||
FROM pending_messages
|
||||
WHERE status = 'processing'
|
||||
GROUP BY session_db_id
|
||||
ORDER BY count DESC
|
||||
`).all();
|
||||
|
||||
if (stuckMessages.length > 0) {
|
||||
console.log('⚠️ Stuck Messages (status=processing):\n');
|
||||
for (const stuck of stuckMessages) {
|
||||
const ageDays = Math.round((Date.now() - stuck.earliest) / (1000 * 60 * 60 * 24));
|
||||
console.log(` Session ${stuck.session_db_id}: ${stuck.count} messages`);
|
||||
console.log(` Stuck for ${ageDays} days (${formatTimestamp(stuck.earliest)})`);
|
||||
}
|
||||
console.log('\n 💡 These will be processed with original timestamps when orphan processing is enabled\n');
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Error:', error);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
main();
|
||||
Executable
+144
@@ -0,0 +1,144 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Verify Timestamp Fix
|
||||
*
|
||||
* This script verifies that the timestamp corruption has been properly fixed.
|
||||
* It checks for any remaining observations in the bad window that shouldn't be there.
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
||||
|
||||
// Bad window: Dec 24 19:45-20:31 (using actual epoch format from database)
|
||||
const BAD_WINDOW_START = 1766623500000; // Dec 24 19:45 PST
|
||||
const BAD_WINDOW_END = 1766626260000; // Dec 24 20:31 PST
|
||||
|
||||
// Original corruption window: Dec 16-22 (when sessions actually started)
|
||||
const ORIGINAL_WINDOW_START = 1765914000000; // Dec 16 00:00 PST
|
||||
const ORIGINAL_WINDOW_END = 1766613600000; // Dec 23 23:59 PST
|
||||
|
||||
interface Observation {
|
||||
id: number;
|
||||
sdk_session_id: string;
|
||||
created_at_epoch: number;
|
||||
created_at: string;
|
||||
title: string;
|
||||
}
|
||||
|
||||
function formatTimestamp(epoch: number): string {
|
||||
return new Date(epoch).toLocaleString('en-US', {
|
||||
timeZone: 'America/Los_Angeles',
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
second: '2-digit'
|
||||
});
|
||||
}
|
||||
|
||||
function main() {
|
||||
console.log('🔍 Verifying timestamp fix...\n');
|
||||
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Check 1: Observations still in bad window
|
||||
console.log('Check 1: Looking for observations still in bad window (Dec 24 19:45-20:31)...');
|
||||
const badWindowObs = db.query<Observation, []>(`
|
||||
SELECT id, sdk_session_id, created_at_epoch, created_at, title
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ${BAD_WINDOW_START}
|
||||
AND created_at_epoch <= ${BAD_WINDOW_END}
|
||||
ORDER BY id
|
||||
`).all();
|
||||
|
||||
if (badWindowObs.length === 0) {
|
||||
console.log('✅ No observations found in bad window - GOOD!\n');
|
||||
} else {
|
||||
console.log(`⚠️ Found ${badWindowObs.length} observations still in bad window:\n`);
|
||||
for (const obs of badWindowObs) {
|
||||
console.log(` Observation #${obs.id}: ${obs.title || '(no title)'}`);
|
||||
console.log(` Timestamp: ${formatTimestamp(obs.created_at_epoch)}`);
|
||||
console.log(` Session: ${obs.sdk_session_id}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
// Check 2: Observations now in original window
|
||||
console.log('Check 2: Counting observations in original window (Dec 17-20)...');
|
||||
const originalWindowObs = db.query<{ count: number }, []>(`
|
||||
SELECT COUNT(*) as count
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ${ORIGINAL_WINDOW_START}
|
||||
AND created_at_epoch <= ${ORIGINAL_WINDOW_END}
|
||||
`).get();
|
||||
|
||||
console.log(`Found ${originalWindowObs?.count || 0} observations in Dec 17-20 window`);
|
||||
console.log('(These should be the corrected observations)\n');
|
||||
|
||||
// Check 3: Session distribution
|
||||
console.log('Check 3: Session distribution of corrected observations...');
|
||||
const sessionDist = db.query<{ sdk_session_id: string; count: number }, []>(`
|
||||
SELECT sdk_session_id, COUNT(*) as count
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ${ORIGINAL_WINDOW_START}
|
||||
AND created_at_epoch <= ${ORIGINAL_WINDOW_END}
|
||||
GROUP BY sdk_session_id
|
||||
ORDER BY count DESC
|
||||
`).all();
|
||||
|
||||
if (sessionDist.length > 0) {
|
||||
console.log(`Observations distributed across ${sessionDist.length} sessions:\n`);
|
||||
for (const dist of sessionDist.slice(0, 10)) {
|
||||
console.log(` ${dist.sdk_session_id}: ${dist.count} observations`);
|
||||
}
|
||||
if (sessionDist.length > 10) {
|
||||
console.log(` ... and ${sessionDist.length - 10} more sessions`);
|
||||
}
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Check 4: Pending messages processed count
|
||||
console.log('Check 4: Verifying processed pending_messages...');
|
||||
const processedCount = db.query<{ count: number }, []>(`
|
||||
SELECT COUNT(*) as count
|
||||
FROM pending_messages
|
||||
WHERE status = 'processed'
|
||||
AND completed_at_epoch >= ${BAD_WINDOW_START}
|
||||
AND completed_at_epoch <= ${BAD_WINDOW_END}
|
||||
`).get();
|
||||
|
||||
console.log(`${processedCount?.count || 0} pending messages were processed during bad window\n`);
|
||||
|
||||
// Summary
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('VERIFICATION SUMMARY:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════\n');
|
||||
|
||||
if (badWindowObs.length === 0 && (originalWindowObs?.count || 0) > 0) {
|
||||
console.log('✅ SUCCESS: Timestamp fix appears to be working correctly!');
|
||||
console.log(` - No observations remain in bad window (Dec 24 19:45-20:31)`);
|
||||
console.log(` - ${originalWindowObs?.count} observations restored to Dec 17-20`);
|
||||
console.log(` - Processed ${processedCount?.count} pending messages`);
|
||||
console.log('\n💡 Safe to re-enable orphan processing in worker-service.ts\n');
|
||||
} else if (badWindowObs.length > 0) {
|
||||
console.log('⚠️ WARNING: Some observations still have incorrect timestamps!');
|
||||
console.log(` - ${badWindowObs.length} observations still in bad window`);
|
||||
console.log(' - Run fix-corrupted-timestamps.ts again or investigate manually\n');
|
||||
} else {
|
||||
console.log('ℹ️ No corrupted observations detected');
|
||||
console.log(' - Either already fixed or corruption never occurred\n');
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Error:', error);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -227,19 +227,20 @@ function main() {
|
||||
claudeSessionToSdkSession.set(sessionMeta.sessionId, existing.sdk_session_id);
|
||||
} else if (existing && !existing.sdk_session_id) {
|
||||
// Session exists but sdk_session_id is NULL, update it
|
||||
const dbId = (db['db'].prepare('SELECT id FROM sdk_sessions WHERE claude_session_id = ?').get(sessionMeta.sessionId) as { id: number }).id;
|
||||
db.updateSDKSessionId(dbId, syntheticSdkSessionId);
|
||||
db['db'].prepare('UPDATE sdk_sessions SET sdk_session_id = ? WHERE claude_session_id = ?')
|
||||
.run(syntheticSdkSessionId, sessionMeta.sessionId);
|
||||
claudeSessionToSdkSession.set(sessionMeta.sessionId, syntheticSdkSessionId);
|
||||
} else {
|
||||
// Create new SDK session
|
||||
const dbId = db.createSDKSession(
|
||||
db.createSDKSession(
|
||||
sessionMeta.sessionId,
|
||||
sessionMeta.project,
|
||||
'Imported from transcript XML'
|
||||
);
|
||||
|
||||
// Update with synthetic SDK session ID
|
||||
db.updateSDKSessionId(dbId, syntheticSdkSessionId);
|
||||
db['db'].prepare('UPDATE sdk_sessions SET sdk_session_id = ? WHERE claude_session_id = ?')
|
||||
.run(syntheticSdkSessionId, sessionMeta.sessionId);
|
||||
|
||||
claudeSessionToSdkSession.set(sessionMeta.sessionId, syntheticSdkSessionId);
|
||||
}
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
import { ProcessManager } from '../services/process/ProcessManager.js';
|
||||
import { getWorkerPort } from '../shared/worker-utils.js';
|
||||
|
||||
const command = process.argv[2];
|
||||
const port = getWorkerPort();
|
||||
|
||||
async function main() {
|
||||
switch (command) {
|
||||
case 'start': {
|
||||
const result = await ProcessManager.start(port);
|
||||
if (result.success) {
|
||||
console.log(`Worker started (PID: ${result.pid})`);
|
||||
const date = new Date().toISOString().slice(0, 10);
|
||||
console.log(`Logs: ~/.claude-mem/logs/worker-${date}.log`);
|
||||
process.exit(0);
|
||||
} else {
|
||||
console.error(`Failed to start: ${result.error}`);
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'stop': {
|
||||
await ProcessManager.stop();
|
||||
console.log('Worker stopped');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
case 'restart': {
|
||||
const result = await ProcessManager.restart(port);
|
||||
if (result.success) {
|
||||
console.log(`Worker restarted (PID: ${result.pid})`);
|
||||
process.exit(0);
|
||||
} else {
|
||||
console.error(`Failed to restart: ${result.error}`);
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'status': {
|
||||
const status = await ProcessManager.status();
|
||||
if (status.running) {
|
||||
console.log('Worker is running');
|
||||
console.log(` PID: ${status.pid}`);
|
||||
console.log(` Port: ${status.port}`);
|
||||
console.log(` Uptime: ${status.uptime}`);
|
||||
} else {
|
||||
console.log('Worker is not running');
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
default:
|
||||
console.log('Usage: worker-cli.js <start|stop|restart|status>');
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch(error => {
|
||||
console.error(error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,68 +0,0 @@
|
||||
/**
|
||||
* Cleanup Hook - SessionEnd
|
||||
*
|
||||
* Pure HTTP client - sends data to worker, worker handles all database operations.
|
||||
* This allows the hook to run under any runtime (Node.js or Bun) since it has no
|
||||
* native module dependencies.
|
||||
*/
|
||||
|
||||
import { stdin } from 'process';
|
||||
import { ensureWorkerRunning, getWorkerPort } from '../shared/worker-utils.js';
|
||||
import { HOOK_TIMEOUTS } from '../shared/hook-constants.js';
|
||||
|
||||
export interface SessionEndInput {
|
||||
session_id: string;
|
||||
reason: 'exit' | 'clear' | 'logout' | 'prompt_input_exit' | 'other';
|
||||
}
|
||||
|
||||
/**
|
||||
* Cleanup Hook Main Logic - Fire-and-forget HTTP client
|
||||
*/
|
||||
async function cleanupHook(input?: SessionEndInput): Promise<void> {
|
||||
// Ensure worker is running before any other logic
|
||||
await ensureWorkerRunning();
|
||||
|
||||
if (!input) {
|
||||
throw new Error('cleanup-hook requires input from Claude Code');
|
||||
}
|
||||
|
||||
const { session_id, reason } = input;
|
||||
|
||||
const port = getWorkerPort();
|
||||
|
||||
// Send to worker - worker handles finding session, marking complete, and stopping spinner
|
||||
const response = await fetch(`http://127.0.0.1:${port}/api/sessions/complete`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
claudeSessionId: session_id,
|
||||
reason
|
||||
}),
|
||||
signal: AbortSignal.timeout(HOOK_TIMEOUTS.DEFAULT)
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Session cleanup failed: ${response.status}`);
|
||||
}
|
||||
|
||||
console.log('{"continue": true, "suppressOutput": true}');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Entry Point
|
||||
if (stdin.isTTY) {
|
||||
// Running manually
|
||||
cleanupHook(undefined);
|
||||
} else {
|
||||
let input = '';
|
||||
stdin.on('data', (chunk) => input += chunk);
|
||||
stdin.on('end', async () => {
|
||||
let parsed: SessionEndInput | undefined;
|
||||
try {
|
||||
parsed = input ? JSON.parse(input) : undefined;
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to parse hook input: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
await cleanupHook(parsed);
|
||||
});
|
||||
}
|
||||
@@ -10,6 +10,7 @@ import { stdin } from "process";
|
||||
import { ensureWorkerRunning, getWorkerPort } from "../shared/worker-utils.js";
|
||||
import { HOOK_TIMEOUTS } from "../shared/hook-constants.js";
|
||||
import { getProjectName } from "../utils/project-name.js";
|
||||
import { logger } from "../utils/logger.js";
|
||||
|
||||
export interface SessionStartInput {
|
||||
session_id: string;
|
||||
|
||||
+11
-2
@@ -2,6 +2,7 @@ import { stdin } from 'process';
|
||||
import { STANDARD_HOOK_RESPONSE } from './hook-response.js';
|
||||
import { ensureWorkerRunning, getWorkerPort } from '../shared/worker-utils.js';
|
||||
import { getProjectName } from '../utils/project-name.js';
|
||||
import { logger } from '../utils/logger.js';
|
||||
|
||||
export interface UserPromptSubmitInput {
|
||||
session_id: string;
|
||||
@@ -24,8 +25,12 @@ async function newHook(input?: UserPromptSubmitInput): Promise<void> {
|
||||
const { session_id, cwd, prompt } = input;
|
||||
const project = getProjectName(cwd);
|
||||
|
||||
logger.info('HOOK', 'new-hook: Received hook input', { session_id, has_prompt: !!prompt, cwd });
|
||||
|
||||
const port = getWorkerPort();
|
||||
|
||||
logger.info('HOOK', 'new-hook: Calling /api/sessions/init', { claudeSessionId: session_id, project, prompt_length: prompt?.length });
|
||||
|
||||
// Initialize session via HTTP - handles DB operations and privacy checks
|
||||
const initResponse = await fetch(`http://127.0.0.1:${port}/api/sessions/init`, {
|
||||
method: 'POST',
|
||||
@@ -46,19 +51,23 @@ async function newHook(input?: UserPromptSubmitInput): Promise<void> {
|
||||
const sessionDbId = initResult.sessionDbId;
|
||||
const promptNumber = initResult.promptNumber;
|
||||
|
||||
logger.info('HOOK', 'new-hook: Received from /api/sessions/init', { sessionDbId, promptNumber, skipped: initResult.skipped });
|
||||
|
||||
// Check if prompt was entirely private (worker performs privacy check)
|
||||
if (initResult.skipped && initResult.reason === 'private') {
|
||||
console.error(`[new-hook] Session ${sessionDbId}, prompt #${promptNumber} (fully private - skipped)`);
|
||||
logger.info('HOOK', `new-hook: Session ${sessionDbId}, prompt #${promptNumber} (fully private - skipped)`);
|
||||
console.log(STANDARD_HOOK_RESPONSE);
|
||||
return;
|
||||
}
|
||||
|
||||
console.error(`[new-hook] Session ${sessionDbId}, prompt #${promptNumber}`);
|
||||
logger.info('HOOK', `new-hook: Session ${sessionDbId}, prompt #${promptNumber}`);
|
||||
|
||||
// Strip leading slash from commands for memory agent
|
||||
// /review 101 → review 101 (more semantic for observations)
|
||||
const cleanedPrompt = prompt.startsWith('/') ? prompt.substring(1) : prompt;
|
||||
|
||||
logger.info('HOOK', 'new-hook: Calling /sessions/{sessionDbId}/init', { sessionDbId, promptNumber, userPrompt_length: cleanedPrompt?.length });
|
||||
|
||||
// Initialize SDK agent session via HTTP (starts the agent!)
|
||||
const response = await fetch(`http://127.0.0.1:${port}/sessions/${sessionDbId}/init`, {
|
||||
method: 'POST',
|
||||
|
||||
@@ -65,6 +65,7 @@ async function summaryHook(input?: StopInput): Promise<void> {
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
console.log(STANDARD_HOOK_RESPONSE);
|
||||
throw new Error(`Summary generation failed: ${response.status}`);
|
||||
}
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
import { basename } from "path";
|
||||
import { ensureWorkerRunning, getWorkerPort } from "../shared/worker-utils.js";
|
||||
import { HOOK_EXIT_CODES } from "../shared/hook-constants.js";
|
||||
import { logger } from "../utils/logger.js";
|
||||
|
||||
// Ensure worker is running
|
||||
await ensureWorkerRunning();
|
||||
|
||||
@@ -6,11 +6,16 @@
|
||||
* Maintains MCP protocol handling and tool schemas
|
||||
*/
|
||||
|
||||
// CRITICAL: Redirect console.log to stderr BEFORE any imports
|
||||
// Import logger first
|
||||
import { logger } from '../utils/logger.js';
|
||||
|
||||
// CRITICAL: Redirect console to stderr BEFORE other imports
|
||||
// MCP uses stdio transport where stdout is reserved for JSON-RPC protocol messages.
|
||||
// Any logs to stdout break the protocol (Claude Desktop parses "[2025..." as JSON array).
|
||||
const _originalConsoleLog = console.log;
|
||||
console.log = (...args: any[]) => console.error(...args);
|
||||
const _originalLog = console['log'];
|
||||
console['log'] = (...args: any[]) => {
|
||||
logger.error('CONSOLE', 'Intercepted console output (MCP protocol protection)', undefined, { args });
|
||||
};
|
||||
|
||||
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
|
||||
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
||||
@@ -18,7 +23,6 @@ import {
|
||||
CallToolRequestSchema,
|
||||
ListToolsRequestSchema,
|
||||
} from '@modelcontextprotocol/sdk/types.js';
|
||||
import { logger } from '../utils/logger.js';
|
||||
import { getWorkerPort, getWorkerHost } from '../shared/worker-utils.js';
|
||||
|
||||
/**
|
||||
@@ -496,7 +500,7 @@ async function main() {
|
||||
if (!workerAvailable) {
|
||||
logger.warn('SYSTEM', 'Worker not available', undefined, { workerUrl: WORKER_BASE_URL });
|
||||
logger.warn('SYSTEM', 'Tools will fail until Worker is started');
|
||||
logger.warn('SYSTEM', 'Start Worker with: claude-mem restart');
|
||||
logger.warn('SYSTEM', 'Start Worker with: npm run worker:restart');
|
||||
} else {
|
||||
logger.info('SYSTEM', 'Worker available', undefined, { workerUrl: WORKER_BASE_URL });
|
||||
}
|
||||
|
||||
@@ -55,6 +55,28 @@ function loadContextConfig(): ContextConfig {
|
||||
const settingsPath = path.join(homedir(), '.claude-mem', 'settings.json');
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
|
||||
// For non-code modes, use all types/concepts from active mode instead of settings
|
||||
const modeId = settings.CLAUDE_MEM_MODE;
|
||||
const isCodeMode = modeId === 'code' || modeId.startsWith('code--');
|
||||
|
||||
let observationTypes: Set<string>;
|
||||
let observationConcepts: Set<string>;
|
||||
|
||||
if (isCodeMode) {
|
||||
// Code mode: use settings-based filtering
|
||||
observationTypes = new Set(
|
||||
settings.CLAUDE_MEM_CONTEXT_OBSERVATION_TYPES.split(',').map((t: string) => t.trim()).filter(Boolean)
|
||||
);
|
||||
observationConcepts = new Set(
|
||||
settings.CLAUDE_MEM_CONTEXT_OBSERVATION_CONCEPTS.split(',').map((c: string) => c.trim()).filter(Boolean)
|
||||
);
|
||||
} else {
|
||||
// Non-code modes: use all types/concepts from active mode
|
||||
const mode = ModeManager.getInstance().getActiveMode();
|
||||
observationTypes = new Set(mode.observation_types.map(t => t.id));
|
||||
observationConcepts = new Set(mode.observation_concepts.map(c => c.id));
|
||||
}
|
||||
|
||||
return {
|
||||
totalObservationCount: parseInt(settings.CLAUDE_MEM_CONTEXT_OBSERVATIONS, 10),
|
||||
fullObservationCount: parseInt(settings.CLAUDE_MEM_CONTEXT_FULL_COUNT, 10),
|
||||
@@ -63,12 +85,8 @@ function loadContextConfig(): ContextConfig {
|
||||
showWorkTokens: settings.CLAUDE_MEM_CONTEXT_SHOW_WORK_TOKENS === 'true',
|
||||
showSavingsAmount: settings.CLAUDE_MEM_CONTEXT_SHOW_SAVINGS_AMOUNT === 'true',
|
||||
showSavingsPercent: settings.CLAUDE_MEM_CONTEXT_SHOW_SAVINGS_PERCENT === 'true',
|
||||
observationTypes: new Set(
|
||||
settings.CLAUDE_MEM_CONTEXT_OBSERVATION_TYPES.split(',').map((t: string) => t.trim()).filter(Boolean)
|
||||
),
|
||||
observationConcepts: new Set(
|
||||
settings.CLAUDE_MEM_CONTEXT_OBSERVATION_CONCEPTS.split(',').map((c: string) => c.trim()).filter(Boolean)
|
||||
),
|
||||
observationTypes,
|
||||
observationConcepts,
|
||||
fullObservationField: settings.CLAUDE_MEM_CONTEXT_FULL_FIELD as 'narrative' | 'facts',
|
||||
showLastSummary: settings.CLAUDE_MEM_CONTEXT_SHOW_LAST_SUMMARY === 'true',
|
||||
showLastMessage: settings.CLAUDE_MEM_CONTEXT_SHOW_LAST_MESSAGE === 'true',
|
||||
@@ -211,7 +229,7 @@ export async function generateContext(input?: ContextInput, useColors: boolean =
|
||||
} catch (unlinkError) {
|
||||
// Marker might not exist
|
||||
}
|
||||
console.error('Native module rebuild needed - restart Claude Code to auto-fix');
|
||||
logger.error('SYSTEM', 'Native module rebuild needed - restart Claude Code to auto-fix');
|
||||
return '';
|
||||
}
|
||||
throw error;
|
||||
|
||||
@@ -1,433 +0,0 @@
|
||||
import { existsSync, readFileSync, writeFileSync, unlinkSync, mkdirSync } from 'fs';
|
||||
import { createWriteStream } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { spawn, spawnSync } from 'child_process';
|
||||
import { homedir } from 'os';
|
||||
import { DATA_DIR } from '../../shared/paths.js';
|
||||
import { getBunPath, isBunAvailable } from '../../utils/bun-path.js';
|
||||
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
|
||||
|
||||
const PID_FILE = join(DATA_DIR, 'worker.pid');
|
||||
const LOG_DIR = join(DATA_DIR, 'logs');
|
||||
const MARKETPLACE_ROOT = join(homedir(), '.claude', 'plugins', 'marketplaces', 'thedotmack');
|
||||
|
||||
interface PidInfo {
|
||||
pid: number;
|
||||
port: number;
|
||||
startedAt: string;
|
||||
version: string;
|
||||
}
|
||||
|
||||
export class ProcessManager {
|
||||
static async start(port: number): Promise<{ success: boolean; pid?: number; error?: string }> {
|
||||
// Validate port range
|
||||
if (isNaN(port) || port < 1024 || port > 65535) {
|
||||
return {
|
||||
success: false,
|
||||
error: `Invalid port ${port}. Must be between 1024 and 65535`
|
||||
};
|
||||
}
|
||||
|
||||
// Check if already running
|
||||
if (await this.isRunning()) {
|
||||
const info = this.getPidInfo();
|
||||
return { success: true, pid: info?.pid };
|
||||
}
|
||||
|
||||
// Ensure log directory exists
|
||||
mkdirSync(LOG_DIR, { recursive: true });
|
||||
|
||||
// On Windows, use the wrapper script to solve zombie port problem
|
||||
// On Unix, use the worker directly
|
||||
const scriptName = process.platform === 'win32' ? 'worker-wrapper.cjs' : 'worker-service.cjs';
|
||||
const workerScript = join(MARKETPLACE_ROOT, 'plugin', 'scripts', scriptName);
|
||||
|
||||
if (!existsSync(workerScript)) {
|
||||
return { success: false, error: `Worker script not found at ${workerScript}` };
|
||||
}
|
||||
|
||||
const logFile = this.getLogFilePath();
|
||||
|
||||
// Use Bun on all platforms with PowerShell workaround for Windows console popups
|
||||
return this.startWithBun(workerScript, logFile, port);
|
||||
}
|
||||
|
||||
private static isBunAvailable(): boolean {
|
||||
return isBunAvailable();
|
||||
}
|
||||
|
||||
/**
|
||||
* Escapes a string for safe use in PowerShell single-quoted strings.
|
||||
* In PowerShell single quotes, the only special character is the single quote itself,
|
||||
* which must be doubled to escape it.
|
||||
*/
|
||||
private static escapePowerShellString(str: string): string {
|
||||
return str.replace(/'/g, "''");
|
||||
}
|
||||
|
||||
private static async startWithBun(script: string, logFile: string, port: number): Promise<{ success: boolean; pid?: number; error?: string }> {
|
||||
const bunPath = getBunPath();
|
||||
if (!bunPath) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'Bun is required but not found in PATH or common installation paths. Install from https://bun.sh'
|
||||
};
|
||||
}
|
||||
try {
|
||||
const isWindows = process.platform === 'win32';
|
||||
|
||||
if (isWindows) {
|
||||
// Windows: Use PowerShell Start-Process with -WindowStyle Hidden
|
||||
// This properly hides the console window (affects both Bun and Node.js)
|
||||
// Note: windowsHide: true doesn't work with detached: true (Bun inherits Node.js process spawning semantics)
|
||||
// See: https://github.com/nodejs/node/issues/21825 and PR #315 for detailed testing
|
||||
//
|
||||
// On Windows, we start worker-wrapper.cjs which manages the actual worker-service.cjs.
|
||||
// This solves the zombie port problem: the wrapper has no sockets, so when it kills
|
||||
// and respawns the inner worker, the socket is properly released.
|
||||
//
|
||||
// Security: All paths (bunPath, script, MARKETPLACE_ROOT) are application-controlled system paths,
|
||||
// not user input. If an attacker could modify these paths, they would already have full filesystem
|
||||
// access including direct access to ~/.claude-mem/claude-mem.db. Nevertheless, we properly escape
|
||||
// all values for PowerShell to follow security best practices.
|
||||
const escapedBunPath = this.escapePowerShellString(bunPath);
|
||||
const escapedScript = this.escapePowerShellString(script);
|
||||
const escapedWorkDir = this.escapePowerShellString(MARKETPLACE_ROOT);
|
||||
const escapedLogFile = this.escapePowerShellString(logFile);
|
||||
const envVars = `$env:CLAUDE_MEM_WORKER_PORT='${port}'`;
|
||||
const psCommand = `${envVars}; Start-Process -FilePath '${escapedBunPath}' -ArgumentList '${escapedScript}' -WorkingDirectory '${escapedWorkDir}' -WindowStyle Hidden -RedirectStandardOutput '${escapedLogFile}' -RedirectStandardError '${escapedLogFile}.err' -PassThru | Select-Object -ExpandProperty Id`;
|
||||
|
||||
const result = spawnSync('powershell', ['-Command', psCommand], {
|
||||
stdio: 'pipe',
|
||||
timeout: 10000,
|
||||
windowsHide: true
|
||||
});
|
||||
|
||||
if (result.status !== 0) {
|
||||
return {
|
||||
success: false,
|
||||
error: `PowerShell spawn failed: ${result.stderr?.toString() || 'unknown error'}`
|
||||
};
|
||||
}
|
||||
|
||||
const pid = parseInt(result.stdout.toString().trim(), 10);
|
||||
if (isNaN(pid)) {
|
||||
return { success: false, error: 'Failed to get PID from PowerShell' };
|
||||
}
|
||||
|
||||
// Write PID file
|
||||
this.writePidFile({
|
||||
pid,
|
||||
port,
|
||||
startedAt: new Date().toISOString(),
|
||||
version: process.env.npm_package_version || 'unknown'
|
||||
});
|
||||
|
||||
// Wait for health
|
||||
return this.waitForHealth(pid, port);
|
||||
} else {
|
||||
// Unix: Use standard spawn with detached
|
||||
const child = spawn(bunPath, [script], {
|
||||
detached: true,
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
env: { ...process.env, CLAUDE_MEM_WORKER_PORT: String(port) },
|
||||
cwd: MARKETPLACE_ROOT
|
||||
});
|
||||
|
||||
// Write logs
|
||||
const logStream = createWriteStream(logFile, { flags: 'a' });
|
||||
child.stdout?.pipe(logStream);
|
||||
child.stderr?.pipe(logStream);
|
||||
|
||||
child.unref();
|
||||
|
||||
if (!child.pid) {
|
||||
return { success: false, error: 'Failed to get PID from spawned process' };
|
||||
}
|
||||
|
||||
// Write PID file
|
||||
this.writePidFile({
|
||||
pid: child.pid,
|
||||
port,
|
||||
startedAt: new Date().toISOString(),
|
||||
version: process.env.npm_package_version || 'unknown'
|
||||
});
|
||||
|
||||
// Wait for health
|
||||
return this.waitForHealth(child.pid, port);
|
||||
}
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
static async stop(timeout: number = 5000): Promise<boolean> {
|
||||
const info = this.getPidInfo();
|
||||
|
||||
if (process.platform === 'win32') {
|
||||
// Windows: Try graceful HTTP shutdown first - this works regardless of PID file state
|
||||
// because the worker shuts itself down from the inside (via wrapper IPC)
|
||||
const port = info?.port ?? this.getPortFromSettings();
|
||||
const httpShutdownSucceeded = await this.tryHttpShutdown(port);
|
||||
|
||||
if (httpShutdownSucceeded) {
|
||||
// HTTP shutdown succeeded - worker confirmed down, safe to remove PID file
|
||||
this.removePidFile();
|
||||
return true;
|
||||
}
|
||||
|
||||
// HTTP shutdown failed (worker not responding), fall back to taskkill
|
||||
if (!info) {
|
||||
// No PID file and HTTP failed - nothing more we can do
|
||||
return true;
|
||||
}
|
||||
|
||||
const { execSync } = await import('child_process');
|
||||
try {
|
||||
// Use taskkill /T /F to kill entire process tree
|
||||
// This ensures the wrapper AND all its children (inner worker, MCP, ChromaSync) are killed
|
||||
// which is necessary to properly release the socket and avoid zombie ports
|
||||
execSync(`taskkill /PID ${info.pid} /T /F`, { timeout: 10000, stdio: 'ignore' });
|
||||
} catch {
|
||||
// Process may already be dead
|
||||
}
|
||||
|
||||
// Wait for process to actually exit before removing PID file
|
||||
try {
|
||||
await this.waitForExit(info.pid, timeout);
|
||||
} catch {
|
||||
// Timeout waiting - process may still be alive
|
||||
}
|
||||
|
||||
// Only remove PID file if process is confirmed dead
|
||||
if (!this.isProcessAlive(info.pid)) {
|
||||
this.removePidFile();
|
||||
}
|
||||
return true;
|
||||
} else {
|
||||
// Unix: Use signals (unchanged behavior)
|
||||
if (!info) return true;
|
||||
|
||||
try {
|
||||
process.kill(info.pid, 'SIGTERM');
|
||||
await this.waitForExit(info.pid, timeout);
|
||||
} catch {
|
||||
try {
|
||||
process.kill(info.pid, 'SIGKILL');
|
||||
} catch {
|
||||
// Process already dead
|
||||
}
|
||||
}
|
||||
|
||||
this.removePidFile();
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
static async restart(port: number): Promise<{ success: boolean; pid?: number; error?: string }> {
|
||||
await this.stop();
|
||||
return this.start(port);
|
||||
}
|
||||
|
||||
static async status(): Promise<{ running: boolean; pid?: number; port?: number; uptime?: string }> {
|
||||
const info = this.getPidInfo();
|
||||
if (!info) return { running: false };
|
||||
|
||||
const running = this.isProcessAlive(info.pid);
|
||||
return {
|
||||
running,
|
||||
pid: running ? info.pid : undefined,
|
||||
port: running ? info.port : undefined,
|
||||
uptime: running ? this.formatUptime(info.startedAt) : undefined
|
||||
};
|
||||
}
|
||||
|
||||
static async isRunning(): Promise<boolean> {
|
||||
const info = this.getPidInfo();
|
||||
if (!info) return false;
|
||||
const alive = this.isProcessAlive(info.pid);
|
||||
if (!alive) {
|
||||
this.removePidFile(); // Clean up stale PID file
|
||||
}
|
||||
return alive;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get worker port from settings file
|
||||
*/
|
||||
private static getPortFromSettings(): number {
|
||||
try {
|
||||
const settingsPath = join(DATA_DIR, 'settings.json');
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
return parseInt(settings.CLAUDE_MEM_WORKER_PORT, 10);
|
||||
} catch {
|
||||
return parseInt(SettingsDefaultsManager.get('CLAUDE_MEM_WORKER_PORT'), 10);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Try to shut down the worker via HTTP endpoint
|
||||
* Returns true if shutdown succeeded, false if worker not responding
|
||||
*/
|
||||
private static async tryHttpShutdown(port: number): Promise<boolean> {
|
||||
try {
|
||||
// Send shutdown request
|
||||
const response = await fetch(`http://127.0.0.1:${port}/api/admin/shutdown`, {
|
||||
method: 'POST',
|
||||
signal: AbortSignal.timeout(2000)
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Wait for worker to actually stop responding
|
||||
return await this.waitForWorkerDown(port, 5000);
|
||||
} catch {
|
||||
// Worker not responding to HTTP - it may be dead or hung
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for worker to stop responding on the given port
|
||||
*/
|
||||
private static async waitForWorkerDown(port: number, timeout: number): Promise<boolean> {
|
||||
const startTime = Date.now();
|
||||
|
||||
while (Date.now() - startTime < timeout) {
|
||||
try {
|
||||
await fetch(`http://127.0.0.1:${port}/api/health`, {
|
||||
signal: AbortSignal.timeout(500)
|
||||
});
|
||||
// Still responding, wait and retry
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
} catch {
|
||||
// Worker stopped responding - success
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
// Timeout - worker still responding
|
||||
return false;
|
||||
}
|
||||
|
||||
// Helper methods
|
||||
private static getPidInfo(): PidInfo | null {
|
||||
try {
|
||||
if (!existsSync(PID_FILE)) return null;
|
||||
const content = readFileSync(PID_FILE, 'utf-8');
|
||||
const parsed = JSON.parse(content);
|
||||
// Validate required fields have correct types
|
||||
if (typeof parsed.pid !== 'number' || typeof parsed.port !== 'number') {
|
||||
logger.warn('PROCESS', 'Malformed PID file: missing or invalid pid/port fields', {}, { parsed });
|
||||
return null;
|
||||
}
|
||||
return parsed as PidInfo;
|
||||
} catch (error) {
|
||||
logger.warn('PROCESS', 'Failed to read PID file', {}, {
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
path: PID_FILE
|
||||
});
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private static writePidFile(info: PidInfo): void {
|
||||
mkdirSync(DATA_DIR, { recursive: true });
|
||||
writeFileSync(PID_FILE, JSON.stringify(info, null, 2));
|
||||
}
|
||||
|
||||
private static removePidFile(): void {
|
||||
try {
|
||||
if (existsSync(PID_FILE)) {
|
||||
unlinkSync(PID_FILE);
|
||||
}
|
||||
} catch {
|
||||
// Ignore errors
|
||||
}
|
||||
}
|
||||
|
||||
private static isProcessAlive(pid: number): boolean {
|
||||
try {
|
||||
process.kill(pid, 0);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private static async waitForHealth(pid: number, port: number, timeoutMs: number = 10000): Promise<{ success: boolean; pid?: number; error?: string }> {
|
||||
const startTime = Date.now();
|
||||
const isWindows = process.platform === 'win32';
|
||||
// Increase timeout on Windows to account for slower process startup
|
||||
const adjustedTimeout = isWindows ? timeoutMs * 2 : timeoutMs;
|
||||
|
||||
while (Date.now() - startTime < adjustedTimeout) {
|
||||
// Check if process is still alive
|
||||
if (!this.isProcessAlive(pid)) {
|
||||
const errorMsg = isWindows
|
||||
? `Process died during startup\n\nTroubleshooting:\n1. Check Task Manager for zombie 'bun.exe' or 'node.exe' processes\n2. Verify port ${port} is not in use: netstat -ano | findstr ${port}\n3. Check worker logs in ~/.claude-mem/logs/\n4. See GitHub issues: #363, #367, #371, #373\n5. Docs: https://docs.claude-mem.ai/troubleshooting/windows-issues`
|
||||
: 'Process died during startup';
|
||||
return { success: false, error: errorMsg };
|
||||
}
|
||||
|
||||
// Try readiness check (changed from /health to /api/readiness)
|
||||
try {
|
||||
const response = await fetch(`http://127.0.0.1:${port}/api/readiness`, {
|
||||
signal: AbortSignal.timeout(1000)
|
||||
});
|
||||
if (response.ok) {
|
||||
return { success: true, pid };
|
||||
}
|
||||
} catch {
|
||||
// Not ready yet, continue polling
|
||||
}
|
||||
|
||||
await new Promise(resolve => setTimeout(resolve, 200));
|
||||
}
|
||||
|
||||
const timeoutMsg = isWindows
|
||||
? `Worker failed to start on Windows (readiness check timed out after ${adjustedTimeout}ms)\n\nTroubleshooting:\n1. Check Task Manager for zombie 'bun.exe' or 'node.exe' processes\n2. Verify port ${port} is not in use: netstat -ano | findstr ${port}\n3. Check worker logs in ~/.claude-mem/logs/\n4. See GitHub issues: #363, #367, #371, #373\n5. Docs: https://docs.claude-mem.ai/troubleshooting/windows-issues`
|
||||
: `Readiness check timed out after ${adjustedTimeout}ms`;
|
||||
|
||||
return { success: false, error: timeoutMsg };
|
||||
}
|
||||
|
||||
private static async waitForExit(pid: number, timeout: number): Promise<void> {
|
||||
const startTime = Date.now();
|
||||
|
||||
while (Date.now() - startTime < timeout) {
|
||||
if (!this.isProcessAlive(pid)) {
|
||||
return;
|
||||
}
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
}
|
||||
|
||||
throw new Error('Process did not exit within timeout');
|
||||
}
|
||||
|
||||
private static getLogFilePath(): string {
|
||||
const date = new Date().toISOString().slice(0, 10);
|
||||
return join(LOG_DIR, `worker-${date}.log`);
|
||||
}
|
||||
|
||||
private static formatUptime(startedAt: string): string {
|
||||
const startTime = new Date(startedAt).getTime();
|
||||
const now = Date.now();
|
||||
const diffMs = now - startTime;
|
||||
|
||||
const seconds = Math.floor(diffMs / 1000);
|
||||
const minutes = Math.floor(seconds / 60);
|
||||
const hours = Math.floor(minutes / 60);
|
||||
const days = Math.floor(hours / 24);
|
||||
|
||||
if (days > 0) return `${days}d ${hours % 24}h`;
|
||||
if (hours > 0) return `${hours}h ${minutes % 60}m`;
|
||||
if (minutes > 0) return `${minutes}m ${seconds % 60}s`;
|
||||
return `${seconds}s`;
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { DATA_DIR, DB_PATH, ensureDir } from '../../shared/paths.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
|
||||
// SQLite configuration constants
|
||||
const SQLITE_MMAP_SIZE_BYTES = 256 * 1024 * 1024; // 256MB
|
||||
@@ -126,7 +127,7 @@ export class DatabaseManager {
|
||||
|
||||
for (const migration of this.migrations) {
|
||||
if (migration.version > maxApplied) {
|
||||
console.log(`Applying migration ${migration.version}...`);
|
||||
logger.info('DB', `Applying migration ${migration.version}`);
|
||||
|
||||
const transaction = this.db.transaction(() => {
|
||||
migration.up(this.db!);
|
||||
@@ -136,7 +137,7 @@ export class DatabaseManager {
|
||||
});
|
||||
|
||||
transaction();
|
||||
console.log(`Migration ${migration.version} applied successfully`);
|
||||
logger.info('DB', `Migration ${migration.version} applied successfully`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { Database } from './sqlite-compat.js';
|
||||
import type { PendingMessage } from '../worker-types.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
|
||||
/**
|
||||
* Persistent pending message record from database
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { TableNameRow } from '../../types/database.js';
|
||||
import { DATA_DIR, DB_PATH, ensureDir } from '../../shared/paths.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import {
|
||||
ObservationSearchResult,
|
||||
SessionSummarySearchResult,
|
||||
@@ -44,9 +45,6 @@ export class SessionSearch {
|
||||
* - Tables maintained but search paths removed
|
||||
* - Triggers still fire to keep tables synchronized
|
||||
*
|
||||
* Note: Using console.log for migration messages since they run during constructor
|
||||
* before structured logger is available. Actual errors use console.error.
|
||||
*
|
||||
* TODO: Remove FTS5 infrastructure in future major version (v7.0.0)
|
||||
*/
|
||||
private ensureFTSTables(): void {
|
||||
@@ -59,7 +57,7 @@ export class SessionSearch {
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[SessionSearch] Creating FTS5 tables...');
|
||||
logger.info('DB', 'Creating FTS5 tables');
|
||||
|
||||
// Create observations_fts virtual table
|
||||
this.db.run(`
|
||||
@@ -143,7 +141,7 @@ export class SessionSearch {
|
||||
END;
|
||||
`);
|
||||
|
||||
console.log('[SessionSearch] FTS5 tables created successfully');
|
||||
logger.info('DB', 'FTS5 tables created successfully');
|
||||
}
|
||||
|
||||
|
||||
@@ -270,7 +268,7 @@ export class SessionSearch {
|
||||
|
||||
// Vector search with query text should be handled by ChromaDB
|
||||
// This method only supports filter-only queries (query=undefined)
|
||||
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
|
||||
logger.warn('DB', 'Text search not supported - use ChromaDB for vector search');
|
||||
return [];
|
||||
}
|
||||
|
||||
@@ -309,7 +307,7 @@ export class SessionSearch {
|
||||
|
||||
// Vector search with query text should be handled by ChromaDB
|
||||
// This method only supports filter-only queries (query=undefined)
|
||||
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
|
||||
logger.warn('DB', 'Text search not supported - use ChromaDB for vector search');
|
||||
return [];
|
||||
}
|
||||
|
||||
@@ -495,7 +493,7 @@ export class SessionSearch {
|
||||
|
||||
// Vector search with query text should be handled by ChromaDB
|
||||
// This method only supports filter-only queries (query=undefined)
|
||||
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
|
||||
logger.warn('DB', 'Text search not supported - use ChromaDB for vector search');
|
||||
return [];
|
||||
}
|
||||
|
||||
|
||||
@@ -20,9 +20,11 @@ import {
|
||||
export class SessionStore {
|
||||
public db: Database;
|
||||
|
||||
constructor() {
|
||||
ensureDir(DATA_DIR);
|
||||
this.db = new Database(DB_PATH);
|
||||
constructor(dbPath: string = DB_PATH) {
|
||||
if (dbPath !== ':memory:') {
|
||||
ensureDir(DATA_DIR);
|
||||
}
|
||||
this.db = new Database(dbPath);
|
||||
|
||||
// Ensure optimized settings
|
||||
this.db.run('PRAGMA journal_mode = WAL');
|
||||
@@ -46,9 +48,6 @@ export class SessionStore {
|
||||
/**
|
||||
* Initialize database schema using migrations (migration004)
|
||||
* This runs the core SDK tables migration if no tables exist
|
||||
*
|
||||
* Note: Using console.log for migration messages since they run during constructor
|
||||
* before structured logger is available. Actual errors use console.error.
|
||||
*/
|
||||
private initializeSchema(): void {
|
||||
try {
|
||||
@@ -68,7 +67,7 @@ export class SessionStore {
|
||||
// Only run migration004 if no migrations have been applied
|
||||
// This creates the sdk_sessions, observations, and session_summaries tables
|
||||
if (maxApplied === 0) {
|
||||
console.log('[SessionStore] Initializing fresh database with migration004...');
|
||||
logger.info('DB', 'Initializing fresh database with migration004');
|
||||
|
||||
// Migration004: SDK agent architecture tables
|
||||
this.db.run(`
|
||||
@@ -132,10 +131,10 @@ export class SessionStore {
|
||||
// Record migration004 as applied
|
||||
this.db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
|
||||
|
||||
console.log('[SessionStore] Migration004 applied successfully');
|
||||
logger.info('DB', 'Migration004 applied successfully');
|
||||
}
|
||||
} catch (error: any) {
|
||||
console.error('[SessionStore] Schema initialization error:', error.message);
|
||||
logger.error('DB', 'Schema initialization error', undefined, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -154,7 +153,7 @@ export class SessionStore {
|
||||
|
||||
if (!hasWorkerPort) {
|
||||
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER');
|
||||
console.log('[SessionStore] Added worker_port column to sdk_sessions table');
|
||||
logger.info('DB', 'Added worker_port column to sdk_sessions table');
|
||||
}
|
||||
|
||||
// Record migration
|
||||
@@ -175,7 +174,7 @@ export class SessionStore {
|
||||
|
||||
if (!hasPromptCounter) {
|
||||
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0');
|
||||
console.log('[SessionStore] Added prompt_counter column to sdk_sessions table');
|
||||
logger.info('DB', 'Added prompt_counter column to sdk_sessions table');
|
||||
}
|
||||
|
||||
// Check observations for prompt_number
|
||||
@@ -184,7 +183,7 @@ export class SessionStore {
|
||||
|
||||
if (!obsHasPromptNumber) {
|
||||
this.db.run('ALTER TABLE observations ADD COLUMN prompt_number INTEGER');
|
||||
console.log('[SessionStore] Added prompt_number column to observations table');
|
||||
logger.info('DB', 'Added prompt_number column to observations table');
|
||||
}
|
||||
|
||||
// Check session_summaries for prompt_number
|
||||
@@ -193,7 +192,7 @@ export class SessionStore {
|
||||
|
||||
if (!sumHasPromptNumber) {
|
||||
this.db.run('ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER');
|
||||
console.log('[SessionStore] Added prompt_number column to session_summaries table');
|
||||
logger.info('DB', 'Added prompt_number column to session_summaries table');
|
||||
}
|
||||
|
||||
// Record migration
|
||||
@@ -218,7 +217,7 @@ export class SessionStore {
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id...');
|
||||
logger.info('DB', 'Removing UNIQUE constraint from session_summaries.sdk_session_id');
|
||||
|
||||
// Begin transaction
|
||||
this.db.run('BEGIN TRANSACTION');
|
||||
@@ -273,7 +272,7 @@ export class SessionStore {
|
||||
// Record migration
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(7, new Date().toISOString());
|
||||
|
||||
console.log('[SessionStore] Successfully removed UNIQUE constraint from session_summaries.sdk_session_id');
|
||||
logger.info('DB', 'Successfully removed UNIQUE constraint from session_summaries.sdk_session_id');
|
||||
} catch (error: any) {
|
||||
// Rollback on error
|
||||
this.db.run('ROLLBACK');
|
||||
@@ -299,7 +298,7 @@ export class SessionStore {
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[SessionStore] Adding hierarchical fields to observations table...');
|
||||
logger.info('DB', 'Adding hierarchical fields to observations table');
|
||||
|
||||
// Add new columns
|
||||
this.db.run(`
|
||||
@@ -315,7 +314,7 @@ export class SessionStore {
|
||||
// Record migration
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(8, new Date().toISOString());
|
||||
|
||||
console.log('[SessionStore] Successfully added hierarchical fields to observations table');
|
||||
logger.info('DB', 'Successfully added hierarchical fields to observations table');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -337,7 +336,7 @@ export class SessionStore {
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[SessionStore] Making observations.text nullable...');
|
||||
logger.info('DB', 'Making observations.text nullable');
|
||||
|
||||
// Begin transaction
|
||||
this.db.run('BEGIN TRANSACTION');
|
||||
@@ -394,7 +393,7 @@ export class SessionStore {
|
||||
// Record migration
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(9, new Date().toISOString());
|
||||
|
||||
console.log('[SessionStore] Successfully made observations.text nullable');
|
||||
logger.info('DB', 'Successfully made observations.text nullable');
|
||||
} catch (error: any) {
|
||||
// Rollback on error
|
||||
this.db.run('ROLLBACK');
|
||||
@@ -418,7 +417,7 @@ export class SessionStore {
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[SessionStore] Creating user_prompts table with FTS5 support...');
|
||||
logger.info('DB', 'Creating user_prompts table with FTS5 support');
|
||||
|
||||
// Begin transaction
|
||||
this.db.run('BEGIN TRANSACTION');
|
||||
@@ -477,7 +476,7 @@ export class SessionStore {
|
||||
// Record migration
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
|
||||
|
||||
console.log('[SessionStore] Successfully created user_prompts table with FTS5 support');
|
||||
logger.info('DB', 'Successfully created user_prompts table with FTS5 support');
|
||||
} catch (error: any) {
|
||||
// Rollback on error
|
||||
this.db.run('ROLLBACK');
|
||||
@@ -502,7 +501,7 @@ export class SessionStore {
|
||||
|
||||
if (!obsHasDiscoveryTokens) {
|
||||
this.db.run('ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0');
|
||||
console.log('[SessionStore] Added discovery_tokens column to observations table');
|
||||
logger.info('DB', 'Added discovery_tokens column to observations table');
|
||||
}
|
||||
|
||||
// Check if discovery_tokens column exists in session_summaries table
|
||||
@@ -511,13 +510,13 @@ export class SessionStore {
|
||||
|
||||
if (!sumHasDiscoveryTokens) {
|
||||
this.db.run('ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0');
|
||||
console.log('[SessionStore] Added discovery_tokens column to session_summaries table');
|
||||
logger.info('DB', 'Added discovery_tokens column to session_summaries table');
|
||||
}
|
||||
|
||||
// Record migration only after successful column verification/addition
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(11, new Date().toISOString());
|
||||
} catch (error: any) {
|
||||
console.error('[SessionStore] Discovery tokens migration error:', error.message);
|
||||
logger.error('DB', 'Discovery tokens migration error', undefined, error);
|
||||
throw error; // Re-throw to prevent silent failures
|
||||
}
|
||||
}
|
||||
@@ -540,7 +539,7 @@ export class SessionStore {
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[SessionStore] Creating pending_messages table...');
|
||||
logger.info('DB', 'Creating pending_messages table');
|
||||
|
||||
this.db.run(`
|
||||
CREATE TABLE pending_messages (
|
||||
@@ -570,9 +569,9 @@ export class SessionStore {
|
||||
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(16, new Date().toISOString());
|
||||
|
||||
console.log('[SessionStore] pending_messages table created successfully');
|
||||
logger.info('DB', 'pending_messages table created successfully');
|
||||
} catch (error: any) {
|
||||
console.error('[SessionStore] Pending messages table migration error:', error.message);
|
||||
logger.error('DB', 'Pending messages table migration error', undefined, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -928,11 +927,13 @@ export class SessionStore {
|
||||
notes: string | null;
|
||||
prompt_number: number | null;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
} | null {
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
files_read, files_edited, notes, prompt_number, created_at,
|
||||
created_at_epoch
|
||||
FROM session_summaries
|
||||
WHERE sdk_session_id = ?
|
||||
ORDER BY created_at_epoch DESC
|
||||
@@ -1037,80 +1038,20 @@ export class SessionStore {
|
||||
return stmt.all(...sdkSessionIds) as any[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Find active SDK session for a Claude session
|
||||
*/
|
||||
findActiveSDKSession(claudeSessionId: string): {
|
||||
id: number;
|
||||
sdk_session_id: string | null;
|
||||
project: string;
|
||||
worker_port: number | null;
|
||||
} | null {
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT id, sdk_session_id, project, worker_port
|
||||
FROM sdk_sessions
|
||||
WHERE claude_session_id = ? AND status = 'active'
|
||||
LIMIT 1
|
||||
`);
|
||||
|
||||
return stmt.get(claudeSessionId) || null;
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/**
|
||||
* Find any SDK session for a Claude session (active, failed, or completed)
|
||||
* Get current prompt number by counting user_prompts for this session
|
||||
* Replaces the prompt_counter column which is no longer maintained
|
||||
*/
|
||||
findAnySDKSession(claudeSessionId: string): { id: number } | null {
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT id
|
||||
FROM sdk_sessions
|
||||
WHERE claude_session_id = ?
|
||||
LIMIT 1
|
||||
`);
|
||||
|
||||
return stmt.get(claudeSessionId) || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reactivate an existing session
|
||||
*/
|
||||
reactivateSession(id: number, userPrompt: string): void {
|
||||
const stmt = this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'active', user_prompt = ?, worker_port = NULL
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(userPrompt, id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Increment prompt counter and return new value
|
||||
*/
|
||||
incrementPromptCounter(id: number): number {
|
||||
const stmt = this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET prompt_counter = COALESCE(prompt_counter, 0) + 1
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(id);
|
||||
|
||||
getPromptNumberFromUserPrompts(claudeSessionId: string): number {
|
||||
const result = this.db.prepare(`
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(id) as { prompt_counter: number } | undefined;
|
||||
|
||||
return result?.prompt_counter || 1;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current prompt counter for a session
|
||||
*/
|
||||
getPromptCounter(id: number): number {
|
||||
const result = this.db.prepare(`
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(id) as { prompt_counter: number | null } | undefined;
|
||||
|
||||
return result?.prompt_counter || 0;
|
||||
SELECT COUNT(*) as count FROM user_prompts WHERE claude_session_id = ?
|
||||
`).get(claudeSessionId) as { count: number };
|
||||
return result.count;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1143,94 +1084,21 @@ export class SessionStore {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
|
||||
// CRITICAL: INSERT OR IGNORE makes this idempotent
|
||||
// First call (prompt #1): Creates new row
|
||||
// Subsequent calls (prompt #2+): Ignored, returns existing ID
|
||||
const stmt = this.db.prepare(`
|
||||
// Pure INSERT OR IGNORE - no updates, no complexity
|
||||
this.db.prepare(`
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`);
|
||||
`).run(claudeSessionId, claudeSessionId, project, userPrompt, now.toISOString(), nowEpoch);
|
||||
|
||||
const result = stmt.run(claudeSessionId, claudeSessionId, project, userPrompt, now.toISOString(), nowEpoch);
|
||||
|
||||
// If lastInsertRowid is 0, insert was ignored (session exists), so fetch existing ID
|
||||
if (result.lastInsertRowid === 0 || result.changes === 0) {
|
||||
// Session exists - UPDATE project and user_prompt if we have non-empty values
|
||||
// This fixes the bug where SAVE hook creates session with empty project,
|
||||
// then NEW hook can't update it because INSERT OR IGNORE skips the insert
|
||||
if (project && project.trim() !== '') {
|
||||
this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET project = ?, user_prompt = ?
|
||||
WHERE claude_session_id = ?
|
||||
`).run(project, userPrompt, claudeSessionId);
|
||||
}
|
||||
|
||||
const selectStmt = this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`);
|
||||
const existing = selectStmt.get(claudeSessionId) as { id: number } | undefined;
|
||||
return existing!.id;
|
||||
}
|
||||
|
||||
return result.lastInsertRowid as number;
|
||||
// Return existing or new ID
|
||||
const row = this.db.prepare('SELECT id FROM sdk_sessions WHERE claude_session_id = ?')
|
||||
.get(claudeSessionId) as { id: number };
|
||||
return row.id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Update SDK session ID (captured from init message)
|
||||
* Only updates if current sdk_session_id is NULL to avoid breaking foreign keys
|
||||
* Returns true if update succeeded, false if skipped
|
||||
*/
|
||||
updateSDKSessionId(id: number, sdkSessionId: string): boolean {
|
||||
const stmt = this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
`);
|
||||
|
||||
const result = stmt.run(sdkSessionId, id);
|
||||
|
||||
if (result.changes === 0) {
|
||||
// This is expected behavior - sdk_session_id is already set
|
||||
// Only log at debug level to avoid noise
|
||||
logger.debug('DB', 'sdk_session_id already set, skipping update', {
|
||||
sessionId: id,
|
||||
sdkSessionId
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Set worker port for a session
|
||||
*/
|
||||
setWorkerPort(id: number, port: number): void {
|
||||
const stmt = this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET worker_port = ?
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(port, id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get worker port for a session
|
||||
*/
|
||||
getWorkerPort(id: number): number | null {
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT worker_port
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
LIMIT 1
|
||||
`);
|
||||
|
||||
const result = stmt.get(id) as { worker_port: number | null } | undefined;
|
||||
return result?.worker_port || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Save a user prompt
|
||||
@@ -1267,7 +1135,7 @@ export class SessionStore {
|
||||
|
||||
/**
|
||||
* Store an observation (from SDK parsing)
|
||||
* Auto-creates session record if it doesn't exist in the index
|
||||
* Assumes session already exists (created by hook)
|
||||
*/
|
||||
storeObservation(
|
||||
sdkSessionId: string,
|
||||
@@ -1283,33 +1151,12 @@ export class SessionStore {
|
||||
files_modified: string[];
|
||||
},
|
||||
promptNumber?: number,
|
||||
discoveryTokens: number = 0
|
||||
discoveryTokens: number = 0,
|
||||
overrideTimestampEpoch?: number
|
||||
): { id: number; createdAtEpoch: number } {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
|
||||
// Ensure session record exists in the index (auto-create if missing)
|
||||
const checkStmt = this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`);
|
||||
const existingSession = checkStmt.get(sdkSessionId) as { id: number } | undefined;
|
||||
|
||||
if (!existingSession) {
|
||||
// Auto-create session record if it doesn't exist
|
||||
const insertSession = this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`);
|
||||
insertSession.run(
|
||||
sdkSessionId, // claude_session_id and sdk_session_id are the same
|
||||
sdkSessionId,
|
||||
project,
|
||||
now.toISOString(),
|
||||
nowEpoch
|
||||
);
|
||||
console.log(`[SessionStore] Auto-created session record for session_id: ${sdkSessionId}`);
|
||||
}
|
||||
// Use override timestamp if provided (for processing backlog messages with original timestamps)
|
||||
const timestampEpoch = overrideTimestampEpoch ?? Date.now();
|
||||
const timestampIso = new Date(timestampEpoch).toISOString();
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
@@ -1331,19 +1178,19 @@ export class SessionStore {
|
||||
JSON.stringify(observation.files_modified),
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
now.toISOString(),
|
||||
nowEpoch
|
||||
timestampIso,
|
||||
timestampEpoch
|
||||
);
|
||||
|
||||
return {
|
||||
id: Number(result.lastInsertRowid),
|
||||
createdAtEpoch: nowEpoch
|
||||
createdAtEpoch: timestampEpoch
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Store a session summary (from SDK parsing)
|
||||
* Auto-creates session record if it doesn't exist in the index
|
||||
* Assumes session already exists - will fail with FK error if not
|
||||
*/
|
||||
storeSummary(
|
||||
sdkSessionId: string,
|
||||
@@ -1357,33 +1204,12 @@ export class SessionStore {
|
||||
notes: string | null;
|
||||
},
|
||||
promptNumber?: number,
|
||||
discoveryTokens: number = 0
|
||||
discoveryTokens: number = 0,
|
||||
overrideTimestampEpoch?: number
|
||||
): { id: number; createdAtEpoch: number } {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
|
||||
// Ensure session record exists in the index (auto-create if missing)
|
||||
const checkStmt = this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`);
|
||||
const existingSession = checkStmt.get(sdkSessionId) as { id: number } | undefined;
|
||||
|
||||
if (!existingSession) {
|
||||
// Auto-create session record if it doesn't exist
|
||||
const insertSession = this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`);
|
||||
insertSession.run(
|
||||
sdkSessionId, // claude_session_id and sdk_session_id are the same
|
||||
sdkSessionId,
|
||||
project,
|
||||
now.toISOString(),
|
||||
nowEpoch
|
||||
);
|
||||
console.log(`[SessionStore] Auto-created session record for session_id: ${sdkSessionId}`);
|
||||
}
|
||||
// Use override timestamp if provided (for processing backlog messages with original timestamps)
|
||||
const timestampEpoch = overrideTimestampEpoch ?? Date.now();
|
||||
const timestampIso = new Date(timestampEpoch).toISOString();
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
@@ -1403,47 +1229,17 @@ export class SessionStore {
|
||||
summary.notes,
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
now.toISOString(),
|
||||
nowEpoch
|
||||
timestampIso,
|
||||
timestampEpoch
|
||||
);
|
||||
|
||||
return {
|
||||
id: Number(result.lastInsertRowid),
|
||||
createdAtEpoch: nowEpoch
|
||||
createdAtEpoch: timestampEpoch
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark SDK session as completed
|
||||
*/
|
||||
markSessionCompleted(id: number): void {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(now.toISOString(), nowEpoch, id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark SDK session as failed
|
||||
*/
|
||||
markSessionFailed(id: number): void {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(now.toISOString(), nowEpoch, id);
|
||||
}
|
||||
|
||||
// REMOVED: cleanupOrphanedSessions - violates "EVERYTHING SHOULD SAVE ALWAYS"
|
||||
// There's no such thing as an "orphaned" session. Sessions are created by hooks
|
||||
@@ -1588,7 +1384,7 @@ export class SessionStore {
|
||||
startEpoch = beforeRecords.length > 0 ? beforeRecords[beforeRecords.length - 1].created_at_epoch : anchorEpoch;
|
||||
endEpoch = afterRecords.length > 0 ? afterRecords[afterRecords.length - 1].created_at_epoch : anchorEpoch;
|
||||
} catch (err: any) {
|
||||
console.error('[SessionStore] Error getting boundary observations:', err.message, project ? `(project: ${project})` : '(all projects)');
|
||||
logger.error('DB', 'Error getting boundary observations', undefined, { error: err, project });
|
||||
return { observations: [], sessions: [], prompts: [] };
|
||||
}
|
||||
} else {
|
||||
@@ -1620,7 +1416,7 @@ export class SessionStore {
|
||||
startEpoch = beforeRecords.length > 0 ? beforeRecords[beforeRecords.length - 1].created_at_epoch : anchorEpoch;
|
||||
endEpoch = afterRecords.length > 0 ? afterRecords[afterRecords.length - 1].created_at_epoch : anchorEpoch;
|
||||
} catch (err: any) {
|
||||
console.error('[SessionStore] Error getting boundary timestamps:', err.message, project ? `(project: ${project})` : '(all projects)');
|
||||
logger.error('DB', 'Error getting boundary timestamps', undefined, { error: err, project });
|
||||
return { observations: [], sessions: [], prompts: [] };
|
||||
}
|
||||
}
|
||||
@@ -1676,7 +1472,7 @@ export class SessionStore {
|
||||
}))
|
||||
};
|
||||
} catch (err: any) {
|
||||
console.error('[SessionStore] Error querying timeline records:', err.message, project ? `(project: ${project})` : '(all projects)');
|
||||
logger.error('DB', 'Error querying timeline records', undefined, { error: err, project });
|
||||
return { observations: [], sessions: [], prompts: [] };
|
||||
}
|
||||
}
|
||||
|
||||
+502
-45
@@ -14,16 +14,203 @@ import { Client } from '@modelcontextprotocol/sdk/client/index.js';
|
||||
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
|
||||
import { getWorkerPort, getWorkerHost } from '../shared/worker-utils.js';
|
||||
import { logger } from '../utils/logger.js';
|
||||
import { exec, execSync } from 'child_process';
|
||||
import { exec, execSync, spawn } from 'child_process';
|
||||
import { homedir } from 'os';
|
||||
import { existsSync, writeFileSync, readFileSync, unlinkSync, mkdirSync } from 'fs';
|
||||
import { promisify } from 'util';
|
||||
|
||||
const execAsync = promisify(exec);
|
||||
|
||||
// PID file management for self-spawn pattern
|
||||
const DATA_DIR = path.join(homedir(), '.claude-mem');
|
||||
const PID_FILE = path.join(DATA_DIR, 'worker.pid');
|
||||
const HOOK_RESPONSE = '{"continue": true, "suppressOutput": true}';
|
||||
|
||||
interface PidInfo {
|
||||
pid: number;
|
||||
port: number;
|
||||
startedAt: string;
|
||||
}
|
||||
|
||||
// PID file utility functions
|
||||
function writePidFile(info: PidInfo): void {
|
||||
mkdirSync(DATA_DIR, { recursive: true });
|
||||
writeFileSync(PID_FILE, JSON.stringify(info, null, 2));
|
||||
}
|
||||
|
||||
function readPidFile(): PidInfo | null {
|
||||
try {
|
||||
if (!existsSync(PID_FILE)) return null;
|
||||
return JSON.parse(readFileSync(PID_FILE, 'utf-8'));
|
||||
} catch (error) {
|
||||
logger.warn('SYSTEM', 'Failed to read PID file', { path: PID_FILE, error: (error as Error).message });
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function removePidFile(): void {
|
||||
try {
|
||||
if (existsSync(PID_FILE)) unlinkSync(PID_FILE);
|
||||
} catch (error) {
|
||||
logger.warn('SYSTEM', 'Failed to remove PID file', { path: PID_FILE, error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
// Lockfile for CLI command mutual exclusion (prevents race conditions on Windows)
|
||||
const LOCK_FILE = path.join(DATA_DIR, 'worker.lock');
|
||||
const LOCK_STALE_MS = 120000; // Lock considered stale after 2 minutes
|
||||
|
||||
interface LockInfo {
|
||||
pid: number;
|
||||
command: string;
|
||||
startedAt: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up stale lock from crashed processes
|
||||
*/
|
||||
function cleanupStaleLock(): void {
|
||||
try {
|
||||
if (!existsSync(LOCK_FILE)) return;
|
||||
const lockData = readFileSync(LOCK_FILE, 'utf-8');
|
||||
const lockInfo: LockInfo = JSON.parse(lockData);
|
||||
const lockAge = Date.now() - new Date(lockInfo.startedAt).getTime();
|
||||
if (lockAge > LOCK_STALE_MS) {
|
||||
logger.warn('SYSTEM', 'Removing stale lock', {
|
||||
lockAge: Math.round(lockAge / 1000) + 's',
|
||||
originalPid: lockInfo.pid,
|
||||
originalCommand: lockInfo.command
|
||||
});
|
||||
unlinkSync(LOCK_FILE);
|
||||
}
|
||||
} catch {
|
||||
// If we can't read the lock, it's likely corrupted - remove it
|
||||
try { unlinkSync(LOCK_FILE); } catch {}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Acquire exclusive lock for worker operations
|
||||
* Uses atomic file creation (O_EXCL) for cross-process safety
|
||||
*/
|
||||
function acquireLock(command: string): boolean {
|
||||
mkdirSync(DATA_DIR, { recursive: true });
|
||||
cleanupStaleLock();
|
||||
|
||||
const lockInfo: LockInfo = {
|
||||
pid: process.pid,
|
||||
command,
|
||||
startedAt: new Date().toISOString()
|
||||
};
|
||||
|
||||
try {
|
||||
// O_EXCL ensures atomic creation - fails if file exists
|
||||
const fd = fs.openSync(LOCK_FILE, fs.constants.O_CREAT | fs.constants.O_EXCL | fs.constants.O_WRONLY);
|
||||
fs.writeSync(fd, JSON.stringify(lockInfo, null, 2));
|
||||
fs.closeSync(fd);
|
||||
return true;
|
||||
} catch (error: unknown) {
|
||||
if ((error as NodeJS.ErrnoException).code === 'EEXIST') {
|
||||
return false;
|
||||
}
|
||||
logger.warn('SYSTEM', 'Lock acquisition error', { error: (error as Error).message });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Release lock file
|
||||
*/
|
||||
function releaseLock(): void {
|
||||
try {
|
||||
if (existsSync(LOCK_FILE)) unlinkSync(LOCK_FILE);
|
||||
} catch (error) {
|
||||
logger.warn('SYSTEM', 'Lock release error', { error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for lock with timeout
|
||||
*/
|
||||
async function waitForLock(command: string, timeoutMs: number): Promise<boolean> {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (acquireLock(command)) return true;
|
||||
await new Promise(r => setTimeout(r, 200));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get platform-adjusted timeout (Windows socket cleanup is slower)
|
||||
*/
|
||||
function getPlatformTimeout(baseMs: number): number {
|
||||
const WINDOWS_MULTIPLIER = 2.0;
|
||||
return process.platform === 'win32' ? Math.round(baseMs * WINDOWS_MULTIPLIER) : baseMs;
|
||||
}
|
||||
|
||||
async function isPortInUse(port: number): Promise<boolean> {
|
||||
try {
|
||||
const response = await fetch(`http://127.0.0.1:${port}/api/health`, {
|
||||
signal: AbortSignal.timeout(2000)
|
||||
});
|
||||
return response.ok;
|
||||
} catch { return false; }
|
||||
}
|
||||
|
||||
async function waitForHealth(port: number, timeoutMs: number = 30000): Promise<boolean> {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
try {
|
||||
const response = await fetch(`http://127.0.0.1:${port}/api/readiness`, {
|
||||
signal: AbortSignal.timeout(2000)
|
||||
});
|
||||
if (response.ok) return true;
|
||||
} catch {
|
||||
// Not ready yet
|
||||
}
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
async function httpShutdown(port: number): Promise<boolean> {
|
||||
try {
|
||||
const response = await fetch(`http://127.0.0.1:${port}/api/admin/shutdown`, {
|
||||
method: 'POST',
|
||||
signal: AbortSignal.timeout(5000)
|
||||
});
|
||||
if (!response.ok) {
|
||||
logger.warn('SYSTEM', 'Shutdown request returned error', { port, status: response.status });
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
} catch (error) {
|
||||
// Connection refused is expected if worker already stopped
|
||||
const isConnectionRefused = (error as Error).message?.includes('ECONNREFUSED');
|
||||
if (!isConnectionRefused) {
|
||||
logger.warn('SYSTEM', 'Shutdown request failed', { port, error: (error as Error).message });
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForPortFree(port: number, timeoutMs: number = 10000): Promise<boolean> {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (!(await isPortInUse(port))) return true;
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// Import composed service layer
|
||||
import { DatabaseManager } from './worker/DatabaseManager.js';
|
||||
import { SessionManager } from './worker/SessionManager.js';
|
||||
import { SSEBroadcaster } from './worker/SSEBroadcaster.js';
|
||||
import { SDKAgent } from './worker/SDKAgent.js';
|
||||
import { GeminiAgent } from './worker/GeminiAgent.js';
|
||||
import { OpenRouterAgent } from './worker/OpenRouterAgent.js';
|
||||
import { PaginationHelper } from './worker/PaginationHelper.js';
|
||||
import { SettingsManager } from './worker/SettingsManager.js';
|
||||
import { SearchManager } from './worker/SearchManager.js';
|
||||
@@ -54,6 +241,8 @@ export class WorkerService {
|
||||
private sessionManager: SessionManager;
|
||||
private sseBroadcaster: SSEBroadcaster;
|
||||
private sdkAgent: SDKAgent;
|
||||
private geminiAgent: GeminiAgent;
|
||||
private openRouterAgent: OpenRouterAgent;
|
||||
private paginationHelper: PaginationHelper;
|
||||
private settingsManager: SettingsManager;
|
||||
private sessionEventBroadcaster: SessionEventBroadcaster;
|
||||
@@ -82,6 +271,10 @@ export class WorkerService {
|
||||
this.sessionManager = new SessionManager(this.dbManager);
|
||||
this.sseBroadcaster = new SSEBroadcaster();
|
||||
this.sdkAgent = new SDKAgent(this.dbManager, this.sessionManager);
|
||||
this.geminiAgent = new GeminiAgent(this.dbManager, this.sessionManager);
|
||||
this.geminiAgent.setFallbackAgent(this.sdkAgent); // Enable fallback to Claude on Gemini API failure
|
||||
this.openRouterAgent = new OpenRouterAgent(this.dbManager, this.sessionManager);
|
||||
this.openRouterAgent.setFallbackAgent(this.sdkAgent); // Enable fallback to Claude on OpenRouter API failure
|
||||
this.paginationHelper = new PaginationHelper(this.dbManager);
|
||||
this.settingsManager = new SettingsManager(this.dbManager);
|
||||
this.sessionEventBroadcaster = new SessionEventBroadcaster(this.sseBroadcaster, this);
|
||||
@@ -99,7 +292,7 @@ export class WorkerService {
|
||||
|
||||
// Initialize route handlers (SearchRoutes will use MCP client initially, then switch to SearchManager after DB init)
|
||||
this.viewerRoutes = new ViewerRoutes(this.sseBroadcaster, this.dbManager, this.sessionManager);
|
||||
this.sessionRoutes = new SessionRoutes(this.sessionManager, this.dbManager, this.sdkAgent, this.sessionEventBroadcaster, this);
|
||||
this.sessionRoutes = new SessionRoutes(this.sessionManager, this.dbManager, this.sdkAgent, this.geminiAgent, this.openRouterAgent, this.sessionEventBroadcaster, this);
|
||||
this.dataRoutes = new DataRoutes(this.paginationHelper, this.dbManager, this.sessionManager, this.sseBroadcaster, this, this.startTime);
|
||||
// SearchRoutes needs SearchManager which requires initialized DB - will be created in initializeBackground()
|
||||
this.searchRoutes = null;
|
||||
@@ -262,7 +455,7 @@ export class WorkerService {
|
||||
this.app.get('/api/context/inject', async (req, res, next) => {
|
||||
try {
|
||||
// Wait for initialization to complete (with timeout)
|
||||
const timeoutMs = 30000; // 30 second timeout
|
||||
const timeoutMs = 300000; // 5 minute timeout for slow systems
|
||||
const timeoutPromise = new Promise((_, reject) =>
|
||||
setTimeout(() => reject(new Error('Initialization timeout')), timeoutMs)
|
||||
);
|
||||
@@ -322,7 +515,7 @@ export class WorkerService {
|
||||
if (isWindows) {
|
||||
// Windows: Use PowerShell Get-CimInstance to find chroma-mcp processes
|
||||
const cmd = `powershell -Command "Get-CimInstance Win32_Process | Where-Object { $_.Name -like '*python*' -and $_.CommandLine -like '*chroma-mcp*' } | Select-Object -ExpandProperty ProcessId"`;
|
||||
const { stdout } = await execAsync(cmd, { timeout: 5000 });
|
||||
const { stdout } = await execAsync(cmd, { timeout: 60000 });
|
||||
|
||||
if (!stdout.trim()) {
|
||||
logger.debug('SYSTEM', 'No orphaned chroma-mcp processes found (Windows)');
|
||||
@@ -377,10 +570,20 @@ export class WorkerService {
|
||||
logger.warn('SYSTEM', 'Skipping invalid PID', { pid });
|
||||
continue;
|
||||
}
|
||||
execSync(`taskkill /PID ${pid} /T /F`, { timeout: 5000, stdio: 'ignore' });
|
||||
try {
|
||||
execSync(`taskkill /PID ${pid} /T /F`, { timeout: 60000, stdio: 'ignore' });
|
||||
} catch {
|
||||
// Process may have already exited - continue cleanup
|
||||
}
|
||||
}
|
||||
} else {
|
||||
await execAsync(`kill ${pids.join(' ')}`);
|
||||
for (const pid of pids) {
|
||||
try {
|
||||
process.kill(pid, 'SIGKILL');
|
||||
} catch {
|
||||
// Process already exited - that's fine
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
logger.info('SYSTEM', 'Orphaned processes cleaned up', { count: pids.length });
|
||||
@@ -427,6 +630,16 @@ export class WorkerService {
|
||||
// Initialize database (once, stays open)
|
||||
await this.dbManager.initialize();
|
||||
|
||||
// Recover stuck messages from previous crashes
|
||||
// Messages stuck in 'processing' state are reset to 'pending' for reprocessing
|
||||
const { PendingMessageStore } = await import('./sqlite/PendingMessageStore.js');
|
||||
const pendingStore = new PendingMessageStore(this.dbManager.getSessionStore().db, 3);
|
||||
const STUCK_THRESHOLD_MS = 5 * 60 * 1000; // 5 minutes
|
||||
const resetCount = pendingStore.resetStuckMessages(STUCK_THRESHOLD_MS);
|
||||
if (resetCount > 0) {
|
||||
logger.info('SYSTEM', `Recovered ${resetCount} stuck messages from previous session`, { thresholdMinutes: 5 });
|
||||
}
|
||||
|
||||
// Initialize search services (requires initialized database)
|
||||
const formattingService = new FormattingService();
|
||||
const timelineService = new TimelineService();
|
||||
@@ -449,11 +662,11 @@ export class WorkerService {
|
||||
env: process.env
|
||||
});
|
||||
|
||||
// Add timeout guard to prevent hanging on MCP connection (15 seconds)
|
||||
const MCP_INIT_TIMEOUT_MS = 15000;
|
||||
// Add timeout guard to prevent hanging on MCP connection (5 minutes for slow systems)
|
||||
const MCP_INIT_TIMEOUT_MS = 300000;
|
||||
const mcpConnectionPromise = this.mcpClient.connect(transport);
|
||||
const timeoutPromise = new Promise<never>((_, reject) =>
|
||||
setTimeout(() => reject(new Error('MCP connection timeout after 15s')), MCP_INIT_TIMEOUT_MS)
|
||||
setTimeout(() => reject(new Error('MCP connection timeout after 5 minutes')), MCP_INIT_TIMEOUT_MS)
|
||||
);
|
||||
|
||||
await Promise.race([mcpConnectionPromise, timeoutPromise]);
|
||||
@@ -464,6 +677,8 @@ export class WorkerService {
|
||||
this.initializationCompleteFlag = true;
|
||||
this.resolveInitialization();
|
||||
logger.info('SYSTEM', 'Background initialization complete');
|
||||
|
||||
// Note: Auto-recovery of orphaned queues disabled - use /api/pending-queue/process endpoint instead
|
||||
} catch (error) {
|
||||
logger.error('SYSTEM', 'Background initialization failed', {}, error as Error);
|
||||
// Don't resolve - let the promise remain pending so readiness check continues to fail
|
||||
@@ -471,6 +686,78 @@ export class WorkerService {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Process pending session queues
|
||||
* Starts SDK agents for sessions that have pending messages but no active processor
|
||||
* @param sessionLimit Maximum number of sessions to start processing (default: 10)
|
||||
* @returns Info about what was started
|
||||
*/
|
||||
async processPendingQueues(sessionLimit: number = 10): Promise<{
|
||||
totalPendingSessions: number;
|
||||
sessionsStarted: number;
|
||||
sessionsSkipped: number;
|
||||
startedSessionIds: number[];
|
||||
}> {
|
||||
const { PendingMessageStore } = await import('./sqlite/PendingMessageStore.js');
|
||||
const pendingStore = new PendingMessageStore(this.dbManager.getSessionStore().db, 3);
|
||||
const orphanedSessionIds = pendingStore.getSessionsWithPendingMessages();
|
||||
|
||||
const result = {
|
||||
totalPendingSessions: orphanedSessionIds.length,
|
||||
sessionsStarted: 0,
|
||||
sessionsSkipped: 0,
|
||||
startedSessionIds: [] as number[]
|
||||
};
|
||||
|
||||
if (orphanedSessionIds.length === 0) {
|
||||
return result;
|
||||
}
|
||||
|
||||
logger.info('SYSTEM', `Processing up to ${sessionLimit} of ${orphanedSessionIds.length} pending session queues`);
|
||||
|
||||
// Process each session sequentially up to the limit
|
||||
for (const sessionDbId of orphanedSessionIds) {
|
||||
if (result.sessionsStarted >= sessionLimit) {
|
||||
break;
|
||||
}
|
||||
|
||||
try {
|
||||
// Skip if session already has an active generator
|
||||
const existingSession = this.sessionManager.getSession(sessionDbId);
|
||||
if (existingSession?.generatorPromise) {
|
||||
result.sessionsSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Initialize session and start SDK agent
|
||||
const session = this.sessionManager.initializeSession(sessionDbId);
|
||||
|
||||
logger.info('SYSTEM', `Starting processor for session ${sessionDbId}`, {
|
||||
project: session.project,
|
||||
pendingCount: pendingStore.getPendingCount(sessionDbId)
|
||||
});
|
||||
|
||||
// Start SDK agent (non-blocking)
|
||||
session.generatorPromise = this.sdkAgent.startSession(session, this)
|
||||
.finally(() => {
|
||||
session.generatorPromise = null;
|
||||
this.broadcastProcessingStatus();
|
||||
});
|
||||
|
||||
result.sessionsStarted++;
|
||||
result.startedSessionIds.push(sessionDbId);
|
||||
|
||||
// Small delay between sessions to avoid rate limiting
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
} catch (error) {
|
||||
logger.warn('SYSTEM', `Failed to process session ${sessionDbId}`, {}, error as Error);
|
||||
result.sessionsSkipped++;
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract a specific section from instruction content
|
||||
* Used by /api/instructions endpoint for progressive instruction loading
|
||||
@@ -563,13 +850,18 @@ export class WorkerService {
|
||||
return [];
|
||||
}
|
||||
|
||||
const cmd = `powershell -Command "Get-CimInstance Win32_Process | Where-Object { $_.ParentProcessId -eq ${parentPid} } | Select-Object -ExpandProperty ProcessId"`;
|
||||
const { stdout } = await execAsync(cmd, { timeout: 5000 });
|
||||
return stdout
|
||||
.trim()
|
||||
.split('\n')
|
||||
.map(s => parseInt(s.trim(), 10))
|
||||
.filter(n => !isNaN(n) && Number.isInteger(n) && n > 0); // SECURITY: Validate each PID
|
||||
try {
|
||||
const cmd = `powershell -Command "Get-CimInstance Win32_Process | Where-Object { $_.ParentProcessId -eq ${parentPid} } | Select-Object -ExpandProperty ProcessId"`;
|
||||
const { stdout } = await execAsync(cmd, { timeout: 60000 });
|
||||
return stdout
|
||||
.trim()
|
||||
.split('\n')
|
||||
.map(s => parseInt(s.trim(), 10))
|
||||
.filter(n => !isNaN(n) && Number.isInteger(n) && n > 0); // SECURITY: Validate each PID
|
||||
} catch (error) {
|
||||
logger.warn('SYSTEM', 'Failed to enumerate child processes', { parentPid, error: (error as Error).message });
|
||||
return []; // Fail safely - continue shutdown without child process cleanup
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -582,12 +874,17 @@ export class WorkerService {
|
||||
return;
|
||||
}
|
||||
|
||||
if (process.platform === 'win32') {
|
||||
// /T kills entire process tree, /F forces termination
|
||||
await execAsync(`taskkill /PID ${pid} /T /F`, { timeout: 5000 });
|
||||
try {
|
||||
if (process.platform === 'win32') {
|
||||
// /T kills entire process tree, /F forces termination
|
||||
await execAsync(`taskkill /PID ${pid} /T /F`, { timeout: 60000 });
|
||||
} else {
|
||||
process.kill(pid, 'SIGKILL');
|
||||
}
|
||||
logger.info('SYSTEM', 'Killed process', { pid });
|
||||
} else {
|
||||
process.kill(pid, 'SIGKILL');
|
||||
} catch {
|
||||
// Process may have already exited - continue shutdown
|
||||
logger.debug('SYSTEM', 'Process already exited during force kill', { pid });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -599,8 +896,12 @@ export class WorkerService {
|
||||
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
const stillAlive = pids.filter(pid => {
|
||||
process.kill(pid, 0); // Signal 0 checks if process exists - throws if dead
|
||||
return true;
|
||||
try {
|
||||
process.kill(pid, 0);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
});
|
||||
|
||||
if (stillAlive.length === 0) {
|
||||
@@ -649,31 +950,187 @@ export class WorkerService {
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Main Entry Point
|
||||
// CLI Entry Point
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Start the worker service (if running as main module)
|
||||
* Note: Using require.main check for CJS compatibility (build outputs CJS)
|
||||
*/
|
||||
if (require.main === module || !module.parent) {
|
||||
const worker = new WorkerService();
|
||||
async function main() {
|
||||
const command = process.argv[2];
|
||||
const port = getWorkerPort();
|
||||
|
||||
// Graceful shutdown
|
||||
process.on('SIGTERM', async () => {
|
||||
logger.info('SYSTEM', 'Received SIGTERM, shutting down gracefully');
|
||||
await worker.shutdown();
|
||||
process.exit(0);
|
||||
});
|
||||
switch (command) {
|
||||
case 'start': {
|
||||
// Acquire lock BEFORE checking port to prevent race condition
|
||||
// If we can't get lock, another session is spawning - wait for health instead
|
||||
if (!acquireLock('start')) {
|
||||
logger.info('SYSTEM', 'Another session is spawning worker, waiting for health');
|
||||
const healthy = await waitForHealth(port, getPlatformTimeout(30000));
|
||||
if (healthy) {
|
||||
logger.info('SYSTEM', 'Worker healthy, returning success');
|
||||
process.exit(0);
|
||||
}
|
||||
// Still not healthy after wait - try to acquire lock and spawn
|
||||
const gotLock = await waitForLock('start', 5000);
|
||||
if (!gotLock) {
|
||||
logger.error('SYSTEM', 'Failed to acquire lock after timeout');
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
process.on('SIGINT', async () => {
|
||||
logger.info('SYSTEM', 'Received SIGINT, shutting down gracefully');
|
||||
await worker.shutdown();
|
||||
process.exit(0);
|
||||
});
|
||||
try {
|
||||
// Re-check port AFTER acquiring lock
|
||||
if (await isPortInUse(port)) {
|
||||
releaseLock();
|
||||
logger.info('SYSTEM', 'Port already in use, worker already running');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
worker.start().catch((error) => {
|
||||
logger.failure('SYSTEM', 'Worker failed to start', {}, error as Error);
|
||||
process.exit(1);
|
||||
});
|
||||
// Spawn self as daemon
|
||||
const child = spawn(process.execPath, [__filename, '--daemon'], {
|
||||
detached: true,
|
||||
stdio: 'ignore',
|
||||
windowsHide: true,
|
||||
env: { ...process.env, CLAUDE_MEM_WORKER_PORT: String(port) }
|
||||
});
|
||||
|
||||
if (child.pid === undefined) {
|
||||
releaseLock();
|
||||
logger.error('SYSTEM', 'Failed to spawn worker daemon');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
child.unref();
|
||||
|
||||
// Write PID file
|
||||
writePidFile({ pid: child.pid, port, startedAt: new Date().toISOString() });
|
||||
|
||||
// Wait for health with platform-adjusted timeout
|
||||
const healthy = await waitForHealth(port, getPlatformTimeout(30000));
|
||||
releaseLock();
|
||||
|
||||
if (!healthy) {
|
||||
removePidFile();
|
||||
logger.error('SYSTEM', 'Worker failed to start');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
logger.info('SYSTEM', 'Worker started successfully');
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
releaseLock();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
case 'stop': {
|
||||
// Acquire lock for stop operation
|
||||
if (!acquireLock('stop')) {
|
||||
// Wait briefly for concurrent operation to complete
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
}
|
||||
|
||||
try {
|
||||
await httpShutdown(port);
|
||||
await waitForPortFree(port, getPlatformTimeout(15000));
|
||||
removePidFile();
|
||||
releaseLock();
|
||||
logger.info('SYSTEM', 'Worker stopped successfully');
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
releaseLock();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
case 'restart': {
|
||||
// Acquire lock for restart operation
|
||||
if (!acquireLock('restart')) {
|
||||
// Another session is already restarting - wait for health
|
||||
logger.info('SYSTEM', 'Another session is restarting worker, waiting');
|
||||
const healthy = await waitForHealth(port, getPlatformTimeout(45000));
|
||||
if (healthy) {
|
||||
logger.info('SYSTEM', 'Worker healthy after restart');
|
||||
process.exit(0);
|
||||
}
|
||||
logger.error('SYSTEM', 'Worker failed to restart (concurrent operation)');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
try {
|
||||
await httpShutdown(port);
|
||||
await waitForPortFree(port, getPlatformTimeout(15000));
|
||||
removePidFile();
|
||||
|
||||
const child = spawn(process.execPath, [__filename, '--daemon'], {
|
||||
detached: true,
|
||||
stdio: 'ignore',
|
||||
windowsHide: true,
|
||||
env: { ...process.env, CLAUDE_MEM_WORKER_PORT: String(port) }
|
||||
});
|
||||
|
||||
if (child.pid === undefined) {
|
||||
releaseLock();
|
||||
logger.error('SYSTEM', 'Failed to spawn worker daemon during restart');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
child.unref();
|
||||
writePidFile({ pid: child.pid, port, startedAt: new Date().toISOString() });
|
||||
|
||||
const healthy = await waitForHealth(port, getPlatformTimeout(30000));
|
||||
releaseLock();
|
||||
|
||||
if (!healthy) {
|
||||
removePidFile();
|
||||
logger.error('SYSTEM', 'Worker failed to restart');
|
||||
process.exit(1);
|
||||
}
|
||||
logger.info('SYSTEM', 'Worker restarted successfully');
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
releaseLock();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
case 'status': {
|
||||
const running = await isPortInUse(port);
|
||||
const pidInfo = readPidFile();
|
||||
if (running && pidInfo) {
|
||||
logger.info('SYSTEM', `Worker running (PID: ${pidInfo.pid}, Port: ${pidInfo.port})`);
|
||||
} else {
|
||||
logger.info('SYSTEM', 'Worker not running');
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
case '--daemon':
|
||||
default: {
|
||||
// Run server directly
|
||||
const worker = new WorkerService();
|
||||
|
||||
process.on('SIGTERM', async () => {
|
||||
logger.info('SYSTEM', 'Received SIGTERM');
|
||||
await worker.shutdown();
|
||||
removePidFile();
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
process.on('SIGINT', async () => {
|
||||
logger.info('SYSTEM', 'Received SIGINT');
|
||||
await worker.shutdown();
|
||||
removePidFile();
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
worker.start().catch((error) => {
|
||||
logger.failure('SYSTEM', 'Worker failed to start', {}, error as Error);
|
||||
removePidFile();
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (require.main === module || !module.parent) {
|
||||
main();
|
||||
}
|
||||
|
||||
@@ -8,6 +8,15 @@ import type { Response } from 'express';
|
||||
// Active Session Types
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Provider-agnostic conversation message for shared history
|
||||
* Used to maintain context across Claude↔Gemini provider switches
|
||||
*/
|
||||
export interface ConversationMessage {
|
||||
role: 'user' | 'assistant';
|
||||
content: string;
|
||||
}
|
||||
|
||||
export interface ActiveSession {
|
||||
sessionDbId: number;
|
||||
claudeSessionId: string;
|
||||
@@ -22,6 +31,9 @@ export interface ActiveSession {
|
||||
cumulativeInputTokens: number; // Track input tokens for discovery cost
|
||||
cumulativeOutputTokens: number; // Track output tokens for discovery cost
|
||||
pendingProcessingIds: Set<number>; // Track ALL message IDs yielded but not yet processed
|
||||
earliestPendingTimestamp: number | null; // Original timestamp of earliest pending message (for accurate observation timestamps)
|
||||
conversationHistory: ConversationMessage[]; // Shared conversation history for provider switching
|
||||
currentProvider: 'claude' | 'gemini' | 'openrouter' | null; // Track which provider is currently running
|
||||
}
|
||||
|
||||
export interface PendingMessage {
|
||||
|
||||
@@ -1,157 +0,0 @@
|
||||
/**
|
||||
* Worker Wrapper - Manages worker process lifecycle
|
||||
*
|
||||
* This wrapper exists to solve the Windows zombie port problem.
|
||||
* The wrapper spawns the actual worker as a child process.
|
||||
* When shutdown is requested, the wrapper kills the child and exits.
|
||||
* The hooks will start a fresh wrapper+worker if needed.
|
||||
*
|
||||
* The wrapper itself has no sockets, so Bun's socket cleanup bug
|
||||
* doesn't affect it.
|
||||
*
|
||||
* NOTE: The wrapper does NOT auto-restart the worker on crash.
|
||||
* This is intentional - the hooks handle startup via ensureWorkerRunning().
|
||||
* Auto-restart would cause PID file mismatches and potential infinite loops.
|
||||
*/
|
||||
|
||||
import { spawn, ChildProcess, execSync } from 'child_process';
|
||||
import path from 'path';
|
||||
|
||||
const isWindows = process.platform === 'win32';
|
||||
|
||||
const SCRIPT_DIR = __dirname;
|
||||
const INNER_SCRIPT = path.join(SCRIPT_DIR, 'worker-service.cjs');
|
||||
|
||||
let inner: ChildProcess | null = null;
|
||||
let isShuttingDown = false;
|
||||
|
||||
function log(msg: string) {
|
||||
const timestamp = new Date().toISOString();
|
||||
console.log(`[${timestamp}] [wrapper] ${msg}`);
|
||||
}
|
||||
|
||||
function spawnInner() {
|
||||
log(`Spawning inner worker: ${INNER_SCRIPT}`);
|
||||
|
||||
inner = spawn(process.execPath, [INNER_SCRIPT], {
|
||||
stdio: ['inherit', 'inherit', 'inherit', 'ipc'],
|
||||
env: { ...process.env, CLAUDE_MEM_MANAGED: 'true' },
|
||||
cwd: path.dirname(INNER_SCRIPT),
|
||||
});
|
||||
|
||||
inner.on('message', async (msg: { type: string }) => {
|
||||
if (msg.type === 'restart' || msg.type === 'shutdown') {
|
||||
// Both restart and shutdown: kill inner and exit wrapper
|
||||
// The hooks will start a fresh wrapper+inner if needed
|
||||
log(`${msg.type} requested by inner`);
|
||||
isShuttingDown = true;
|
||||
await killInner();
|
||||
log('Exiting wrapper');
|
||||
process.exit(0);
|
||||
}
|
||||
});
|
||||
|
||||
inner.on('exit', (code, signal) => {
|
||||
log(`Inner exited with code=${code}, signal=${signal}`);
|
||||
inner = null;
|
||||
|
||||
// Don't auto-restart - let hooks handle it via ensureWorkerRunning()
|
||||
// Auto-restart causes PID file mismatches and potential infinite loops
|
||||
if (!isShuttingDown) {
|
||||
log('Inner exited unexpectedly, wrapper exiting (hooks will restart if needed)');
|
||||
process.exit(code ?? 1);
|
||||
}
|
||||
});
|
||||
|
||||
inner.on('error', (err) => {
|
||||
log(`Inner error: ${err.message}`);
|
||||
});
|
||||
}
|
||||
|
||||
async function killInner(): Promise<void> {
|
||||
if (!inner || !inner.pid) {
|
||||
log('No inner process to kill');
|
||||
return;
|
||||
}
|
||||
|
||||
const pid = inner.pid;
|
||||
log(`Killing inner process tree (pid=${pid})`);
|
||||
|
||||
if (isWindows) {
|
||||
// On Windows, use taskkill /T /F to kill entire process tree
|
||||
// This ensures all children (MCP server, ChromaSync, etc.) are killed
|
||||
// which is necessary to properly release the socket
|
||||
try {
|
||||
execSync(`taskkill /PID ${pid} /T /F`, { timeout: 10000, stdio: 'ignore' });
|
||||
log(`taskkill completed for pid=${pid}`);
|
||||
} catch (error) {
|
||||
// Process may already be dead
|
||||
log(`taskkill failed (process may be dead): ${error}`);
|
||||
}
|
||||
} else {
|
||||
// On Unix, SIGTERM then SIGKILL
|
||||
inner.kill('SIGTERM');
|
||||
|
||||
// Wait for exit with timeout
|
||||
const exitPromise = new Promise<void>(resolve => {
|
||||
if (!inner) {
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
inner.on('exit', () => resolve());
|
||||
});
|
||||
|
||||
const timeoutPromise = new Promise<void>(resolve =>
|
||||
setTimeout(() => resolve(), 5000)
|
||||
);
|
||||
|
||||
await Promise.race([exitPromise, timeoutPromise]);
|
||||
|
||||
// Force kill if still alive
|
||||
if (inner && !inner.killed) {
|
||||
log('Inner did not exit gracefully, force killing');
|
||||
inner.kill('SIGKILL');
|
||||
}
|
||||
}
|
||||
|
||||
// Wait for the process to fully exit
|
||||
await waitForProcessExit(pid, 5000);
|
||||
|
||||
inner = null;
|
||||
log('Inner process terminated');
|
||||
}
|
||||
|
||||
async function waitForProcessExit(pid: number, timeoutMs: number): Promise<void> {
|
||||
const start = Date.now();
|
||||
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
try {
|
||||
process.kill(pid, 0); // Check if process exists
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
} catch {
|
||||
// Process is dead
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
log(`Timeout waiting for process ${pid} to exit`);
|
||||
}
|
||||
|
||||
// Handle wrapper signals
|
||||
process.on('SIGTERM', async () => {
|
||||
log('Wrapper received SIGTERM');
|
||||
isShuttingDown = true;
|
||||
await killInner();
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
process.on('SIGINT', async () => {
|
||||
log('Wrapper received SIGINT');
|
||||
isShuttingDown = true;
|
||||
await killInner();
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
// Start the inner worker
|
||||
log('Wrapper starting');
|
||||
spawnInner();
|
||||
@@ -28,9 +28,9 @@ function isValidBranchName(branchName: string): boolean {
|
||||
return validBranchRegex.test(branchName) && !branchName.includes('..');
|
||||
}
|
||||
|
||||
// Timeout constants
|
||||
const GIT_COMMAND_TIMEOUT_MS = 30_000;
|
||||
const NPM_INSTALL_TIMEOUT_MS = 120_000;
|
||||
// Timeout constants (increased for slow systems)
|
||||
const GIT_COMMAND_TIMEOUT_MS = 300_000;
|
||||
const NPM_INSTALL_TIMEOUT_MS = 600_000;
|
||||
const DEFAULT_SHELL_TIMEOUT_MS = 60_000;
|
||||
|
||||
export interface BranchInfo {
|
||||
|
||||
@@ -31,7 +31,9 @@ export class DatabaseManager {
|
||||
this.chromaSync = new ChromaSync('claude-mem');
|
||||
|
||||
// Start background backfill (fire-and-forget)
|
||||
this.chromaSync.ensureBackfilled();
|
||||
this.chromaSync.ensureBackfilled().catch(error => {
|
||||
logger.error('DB', 'Chroma backfill failed (non-fatal)', {}, error);
|
||||
});
|
||||
|
||||
logger.info('DB', 'Database initialized');
|
||||
}
|
||||
@@ -108,10 +110,4 @@ export class DatabaseManager {
|
||||
return session;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark session as completed
|
||||
*/
|
||||
markSessionComplete(sessionDbId: number): void {
|
||||
this.getSessionStore().markSessionCompleted(sessionDbId);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
|
||||
import { ObservationSearchResult, SessionSummarySearchResult, UserPromptSearchResult } from '../sqlite/types.js';
|
||||
import { ModeManager } from '../domain/ModeManager.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
|
||||
// Token estimation constant (matches context-generator)
|
||||
const CHARS_PER_TOKEN_ESTIMATE = 4;
|
||||
|
||||
@@ -0,0 +1,583 @@
|
||||
/**
|
||||
* GeminiAgent: Gemini-based observation extraction
|
||||
*
|
||||
* Alternative to SDKAgent that uses Google's Gemini API directly
|
||||
* for extracting observations from tool usage.
|
||||
*
|
||||
* Responsibility:
|
||||
* - Call Gemini REST API for observation extraction
|
||||
* - Parse XML responses (same format as Claude)
|
||||
* - Sync to database and Chroma
|
||||
*/
|
||||
|
||||
import path from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { DatabaseManager } from './DatabaseManager.js';
|
||||
import { SessionManager } from './SessionManager.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { parseObservations, parseSummary } from '../../sdk/parser.js';
|
||||
import { buildInitPrompt, buildObservationPrompt, buildSummaryPrompt, buildContinuationPrompt } from '../../sdk/prompts.js';
|
||||
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
|
||||
import type { ActiveSession, ConversationMessage } from '../worker-types.js';
|
||||
import { ModeManager } from '../domain/ModeManager.js';
|
||||
|
||||
// Gemini API endpoint
|
||||
const GEMINI_API_URL = 'https://generativelanguage.googleapis.com/v1beta/models';
|
||||
|
||||
// Gemini model types (available via API)
|
||||
export type GeminiModel =
|
||||
| 'gemini-2.5-flash-lite'
|
||||
| 'gemini-2.5-flash'
|
||||
| 'gemini-2.5-pro'
|
||||
| 'gemini-2.0-flash'
|
||||
| 'gemini-2.0-flash-lite';
|
||||
|
||||
// Free tier RPM limits by model (requests per minute)
|
||||
const GEMINI_RPM_LIMITS: Record<GeminiModel, number> = {
|
||||
'gemini-2.5-flash-lite': 10,
|
||||
'gemini-2.5-flash': 10,
|
||||
'gemini-2.5-pro': 5,
|
||||
'gemini-2.0-flash': 15,
|
||||
'gemini-2.0-flash-lite': 30,
|
||||
};
|
||||
|
||||
// Track last request time for rate limiting
|
||||
let lastRequestTime = 0;
|
||||
|
||||
/**
|
||||
* Enforce RPM rate limit for Gemini free tier.
|
||||
* Waits the required time between requests based on model's RPM limit + 100ms safety buffer.
|
||||
* Skipped entirely if rate limiting is disabled (billing users with 1000+ RPM available).
|
||||
*/
|
||||
async function enforceRateLimitForModel(model: GeminiModel, rateLimitingEnabled: boolean): Promise<void> {
|
||||
// Skip rate limiting if disabled (billing users with 1000+ RPM)
|
||||
if (!rateLimitingEnabled) {
|
||||
return;
|
||||
}
|
||||
|
||||
const rpm = GEMINI_RPM_LIMITS[model] || 5;
|
||||
const minimumDelayMs = Math.ceil(60000 / rpm) + 100; // (60s / RPM) + 100ms safety buffer
|
||||
|
||||
const now = Date.now();
|
||||
const timeSinceLastRequest = now - lastRequestTime;
|
||||
|
||||
if (timeSinceLastRequest < minimumDelayMs) {
|
||||
const waitTime = minimumDelayMs - timeSinceLastRequest;
|
||||
logger.debug('SDK', `Rate limiting: waiting ${waitTime}ms before Gemini request`, { model, rpm });
|
||||
await new Promise(resolve => setTimeout(resolve, waitTime));
|
||||
}
|
||||
|
||||
lastRequestTime = Date.now();
|
||||
}
|
||||
|
||||
interface GeminiResponse {
|
||||
candidates?: Array<{
|
||||
content?: {
|
||||
parts?: Array<{
|
||||
text?: string;
|
||||
}>;
|
||||
};
|
||||
}>;
|
||||
usageMetadata?: {
|
||||
promptTokenCount?: number;
|
||||
candidatesTokenCount?: number;
|
||||
totalTokenCount?: number;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Gemini content message format
|
||||
* role: "user" or "model" (Gemini uses "model" not "assistant")
|
||||
*/
|
||||
interface GeminiContent {
|
||||
role: 'user' | 'model';
|
||||
parts: Array<{ text: string }>;
|
||||
}
|
||||
|
||||
// Forward declaration for fallback agent type
|
||||
type FallbackAgent = {
|
||||
startSession(session: ActiveSession, worker?: any): Promise<void>;
|
||||
};
|
||||
|
||||
export class GeminiAgent {
|
||||
private dbManager: DatabaseManager;
|
||||
private sessionManager: SessionManager;
|
||||
private fallbackAgent: FallbackAgent | null = null;
|
||||
|
||||
constructor(dbManager: DatabaseManager, sessionManager: SessionManager) {
|
||||
this.dbManager = dbManager;
|
||||
this.sessionManager = sessionManager;
|
||||
}
|
||||
|
||||
/**
|
||||
* Set the fallback agent (Claude SDK) for when Gemini API fails
|
||||
* Must be set after construction to avoid circular dependency
|
||||
*/
|
||||
setFallbackAgent(agent: FallbackAgent): void {
|
||||
this.fallbackAgent = agent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if an error should trigger fallback to Claude
|
||||
*/
|
||||
private shouldFallbackToClaude(error: any): boolean {
|
||||
const message = error?.message || '';
|
||||
// Fall back on rate limit (429), server errors (5xx), or network issues
|
||||
return (
|
||||
message.includes('429') ||
|
||||
message.includes('500') ||
|
||||
message.includes('502') ||
|
||||
message.includes('503') ||
|
||||
message.includes('ECONNREFUSED') ||
|
||||
message.includes('ETIMEDOUT') ||
|
||||
message.includes('fetch failed')
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Start Gemini agent for a session
|
||||
* Uses multi-turn conversation to maintain context across messages
|
||||
*/
|
||||
async startSession(session: ActiveSession, worker?: any): Promise<void> {
|
||||
try {
|
||||
// Get Gemini configuration
|
||||
const { apiKey, model, rateLimitingEnabled } = this.getGeminiConfig();
|
||||
|
||||
if (!apiKey) {
|
||||
throw new Error('Gemini API key not configured. Set CLAUDE_MEM_GEMINI_API_KEY in settings or GEMINI_API_KEY environment variable.');
|
||||
}
|
||||
|
||||
// Load active mode
|
||||
const mode = ModeManager.getInstance().getActiveMode();
|
||||
|
||||
// Build initial prompt
|
||||
const initPrompt = session.lastPromptNumber === 1
|
||||
? buildInitPrompt(session.project, session.claudeSessionId, session.userPrompt, mode)
|
||||
: buildContinuationPrompt(session.userPrompt, session.lastPromptNumber, session.claudeSessionId, mode);
|
||||
|
||||
// Add to conversation history and query Gemini with full context
|
||||
session.conversationHistory.push({ role: 'user', content: initPrompt });
|
||||
const initResponse = await this.queryGeminiMultiTurn(session.conversationHistory, apiKey, model, rateLimitingEnabled);
|
||||
|
||||
if (initResponse.content) {
|
||||
// Add response to conversation history
|
||||
session.conversationHistory.push({ role: 'assistant', content: initResponse.content });
|
||||
|
||||
// Track token usage
|
||||
const tokensUsed = initResponse.tokensUsed || 0;
|
||||
session.cumulativeInputTokens += Math.floor(tokensUsed * 0.7); // Rough estimate
|
||||
session.cumulativeOutputTokens += Math.floor(tokensUsed * 0.3);
|
||||
|
||||
// Process response (no original timestamp for init - not from queue)
|
||||
await this.processGeminiResponse(session, initResponse.content, worker, tokensUsed, null);
|
||||
} else {
|
||||
logger.warn('SDK', 'Empty Gemini init response - session may lack context', {
|
||||
sessionId: session.sessionDbId,
|
||||
model
|
||||
});
|
||||
}
|
||||
|
||||
// Process pending messages
|
||||
for await (const message of this.sessionManager.getMessageIterator(session.sessionDbId)) {
|
||||
// Capture earliest timestamp BEFORE processing (will be cleared after)
|
||||
// This ensures backlog messages get their original timestamps, not current time
|
||||
const originalTimestamp = session.earliestPendingTimestamp;
|
||||
|
||||
if (message.type === 'observation') {
|
||||
// Update last prompt number
|
||||
if (message.prompt_number !== undefined) {
|
||||
session.lastPromptNumber = message.prompt_number;
|
||||
}
|
||||
|
||||
// Build observation prompt
|
||||
const obsPrompt = buildObservationPrompt({
|
||||
id: 0,
|
||||
tool_name: message.tool_name!,
|
||||
tool_input: JSON.stringify(message.tool_input),
|
||||
tool_output: JSON.stringify(message.tool_response),
|
||||
created_at_epoch: originalTimestamp ?? Date.now(),
|
||||
cwd: message.cwd
|
||||
});
|
||||
|
||||
// Add to conversation history and query Gemini with full context
|
||||
session.conversationHistory.push({ role: 'user', content: obsPrompt });
|
||||
const obsResponse = await this.queryGeminiMultiTurn(session.conversationHistory, apiKey, model, rateLimitingEnabled);
|
||||
|
||||
if (obsResponse.content) {
|
||||
// Add response to conversation history
|
||||
session.conversationHistory.push({ role: 'assistant', content: obsResponse.content });
|
||||
|
||||
const tokensUsed = obsResponse.tokensUsed || 0;
|
||||
session.cumulativeInputTokens += Math.floor(tokensUsed * 0.7);
|
||||
session.cumulativeOutputTokens += Math.floor(tokensUsed * 0.3);
|
||||
await this.processGeminiResponse(session, obsResponse.content, worker, tokensUsed, originalTimestamp);
|
||||
} else {
|
||||
// Empty response - still mark messages as processed to avoid stuck state
|
||||
logger.warn('SDK', 'Empty Gemini response for observation, marking as processed', {
|
||||
sessionId: session.sessionDbId,
|
||||
toolName: message.tool_name
|
||||
});
|
||||
await this.markMessagesProcessed(session, worker);
|
||||
}
|
||||
|
||||
} else if (message.type === 'summarize') {
|
||||
// Build summary prompt
|
||||
const summaryPrompt = buildSummaryPrompt({
|
||||
id: session.sessionDbId,
|
||||
sdk_session_id: session.sdkSessionId,
|
||||
project: session.project,
|
||||
user_prompt: session.userPrompt,
|
||||
last_user_message: message.last_user_message || '',
|
||||
last_assistant_message: message.last_assistant_message || ''
|
||||
}, mode);
|
||||
|
||||
// Add to conversation history and query Gemini with full context
|
||||
session.conversationHistory.push({ role: 'user', content: summaryPrompt });
|
||||
const summaryResponse = await this.queryGeminiMultiTurn(session.conversationHistory, apiKey, model, rateLimitingEnabled);
|
||||
|
||||
if (summaryResponse.content) {
|
||||
// Add response to conversation history
|
||||
session.conversationHistory.push({ role: 'assistant', content: summaryResponse.content });
|
||||
|
||||
const tokensUsed = summaryResponse.tokensUsed || 0;
|
||||
session.cumulativeInputTokens += Math.floor(tokensUsed * 0.7);
|
||||
session.cumulativeOutputTokens += Math.floor(tokensUsed * 0.3);
|
||||
await this.processGeminiResponse(session, summaryResponse.content, worker, tokensUsed, originalTimestamp);
|
||||
} else {
|
||||
// Empty response - still mark messages as processed to avoid stuck state
|
||||
logger.warn('SDK', 'Empty Gemini response for summary, marking as processed', {
|
||||
sessionId: session.sessionDbId
|
||||
});
|
||||
await this.markMessagesProcessed(session, worker);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Mark session complete
|
||||
const sessionDuration = Date.now() - session.startTime;
|
||||
logger.success('SDK', 'Gemini agent completed', {
|
||||
sessionId: session.sessionDbId,
|
||||
duration: `${(sessionDuration / 1000).toFixed(1)}s`,
|
||||
historyLength: session.conversationHistory.length
|
||||
});
|
||||
|
||||
} catch (error: any) {
|
||||
if (error.name === 'AbortError') {
|
||||
logger.warn('SDK', 'Gemini agent aborted', { sessionId: session.sessionDbId });
|
||||
throw error;
|
||||
}
|
||||
|
||||
// Check if we should fall back to Claude
|
||||
if (this.shouldFallbackToClaude(error) && this.fallbackAgent) {
|
||||
logger.warn('SDK', 'Gemini API failed, falling back to Claude SDK', {
|
||||
sessionDbId: session.sessionDbId,
|
||||
error: error.message,
|
||||
historyLength: session.conversationHistory.length
|
||||
});
|
||||
|
||||
// Reset any 'processing' messages back to 'pending' so Claude can retry them
|
||||
// This handles the case where Gemini failed mid-processing a message
|
||||
const pendingStore = this.sessionManager.getPendingMessageStore();
|
||||
const resetCount = pendingStore.resetStuckMessages(0); // 0 = reset ALL processing messages
|
||||
if (resetCount > 0) {
|
||||
logger.info('SDK', 'Reset processing messages for fallback', {
|
||||
sessionDbId: session.sessionDbId,
|
||||
resetCount
|
||||
});
|
||||
}
|
||||
|
||||
// Fall back to Claude - it will use the same session with shared conversationHistory
|
||||
// Note: Claude SDK will continue processing from current state
|
||||
return this.fallbackAgent.startSession(session, worker);
|
||||
}
|
||||
|
||||
logger.failure('SDK', 'Gemini agent error', { sessionDbId: session.sessionDbId }, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert shared ConversationMessage array to Gemini's contents format
|
||||
* Maps 'assistant' role to 'model' for Gemini API compatibility
|
||||
*/
|
||||
private conversationToGeminiContents(history: ConversationMessage[]): GeminiContent[] {
|
||||
return history.map(msg => ({
|
||||
role: msg.role === 'assistant' ? 'model' : 'user',
|
||||
parts: [{ text: msg.content }]
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Query Gemini via REST API with full conversation history (multi-turn)
|
||||
* Sends the entire conversation context for coherent responses
|
||||
*/
|
||||
private async queryGeminiMultiTurn(
|
||||
history: ConversationMessage[],
|
||||
apiKey: string,
|
||||
model: GeminiModel,
|
||||
rateLimitingEnabled: boolean
|
||||
): Promise<{ content: string; tokensUsed?: number }> {
|
||||
const contents = this.conversationToGeminiContents(history);
|
||||
const totalChars = history.reduce((sum, m) => sum + m.content.length, 0);
|
||||
|
||||
logger.debug('SDK', `Querying Gemini multi-turn (${model})`, {
|
||||
turns: history.length,
|
||||
totalChars
|
||||
});
|
||||
|
||||
const url = `${GEMINI_API_URL}/${model}:generateContent?key=${apiKey}`;
|
||||
|
||||
// Enforce RPM rate limit for free tier (skipped if rate limiting disabled)
|
||||
await enforceRateLimitForModel(model, rateLimitingEnabled);
|
||||
|
||||
const response = await fetch(url, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
contents,
|
||||
generationConfig: {
|
||||
temperature: 0.3, // Lower temperature for structured extraction
|
||||
maxOutputTokens: 4096,
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.text();
|
||||
throw new Error(`Gemini API error: ${response.status} - ${error}`);
|
||||
}
|
||||
|
||||
const data = await response.json() as GeminiResponse;
|
||||
|
||||
if (!data.candidates?.[0]?.content?.parts?.[0]?.text) {
|
||||
logger.warn('SDK', 'Empty response from Gemini');
|
||||
return { content: '' };
|
||||
}
|
||||
|
||||
const content = data.candidates[0].content.parts[0].text;
|
||||
const tokensUsed = data.usageMetadata?.totalTokenCount;
|
||||
|
||||
return { content, tokensUsed };
|
||||
}
|
||||
|
||||
/**
|
||||
* Process Gemini response (same format as Claude)
|
||||
* @param originalTimestamp - Original epoch when message was queued (for backlog processing accuracy)
|
||||
*/
|
||||
private async processGeminiResponse(
|
||||
session: ActiveSession,
|
||||
text: string,
|
||||
worker: any | undefined,
|
||||
discoveryTokens: number,
|
||||
originalTimestamp: number | null
|
||||
): Promise<void> {
|
||||
// Parse observations (same XML format)
|
||||
const observations = parseObservations(text, session.claudeSessionId);
|
||||
|
||||
// Store observations with original timestamp (if processing backlog) or current time
|
||||
for (const obs of observations) {
|
||||
const { id: obsId, createdAtEpoch } = this.dbManager.getSessionStore().storeObservation(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens,
|
||||
originalTimestamp ?? undefined
|
||||
);
|
||||
|
||||
logger.info('SDK', 'Gemini observation saved', {
|
||||
sessionId: session.sessionDbId,
|
||||
obsId,
|
||||
type: obs.type,
|
||||
title: obs.title || '(untitled)'
|
||||
});
|
||||
|
||||
// Sync to Chroma
|
||||
this.dbManager.getChromaSync().syncObservation(
|
||||
obsId,
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
createdAtEpoch,
|
||||
discoveryTokens
|
||||
).catch(err => {
|
||||
logger.warn('SDK', 'Gemini chroma sync failed', { obsId }, err);
|
||||
});
|
||||
|
||||
// Broadcast to SSE clients
|
||||
if (worker && worker.sseBroadcaster) {
|
||||
worker.sseBroadcaster.broadcast({
|
||||
type: 'new_observation',
|
||||
observation: {
|
||||
id: obsId,
|
||||
sdk_session_id: session.sdkSessionId,
|
||||
session_id: session.claudeSessionId,
|
||||
type: obs.type,
|
||||
title: obs.title,
|
||||
subtitle: obs.subtitle,
|
||||
text: null,
|
||||
narrative: obs.narrative || null,
|
||||
facts: JSON.stringify(obs.facts || []),
|
||||
concepts: JSON.stringify(obs.concepts || []),
|
||||
files_read: JSON.stringify(obs.files_read || []),
|
||||
files_modified: JSON.stringify(obs.files_modified || []),
|
||||
project: session.project,
|
||||
prompt_number: session.lastPromptNumber,
|
||||
created_at_epoch: createdAtEpoch
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Parse summary
|
||||
const summary = parseSummary(text, session.sessionDbId);
|
||||
|
||||
if (summary) {
|
||||
// Convert nullable fields to empty strings for storeSummary
|
||||
const summaryForStore = {
|
||||
request: summary.request || '',
|
||||
investigated: summary.investigated || '',
|
||||
learned: summary.learned || '',
|
||||
completed: summary.completed || '',
|
||||
next_steps: summary.next_steps || '',
|
||||
notes: summary.notes
|
||||
};
|
||||
|
||||
const { id: summaryId, createdAtEpoch } = this.dbManager.getSessionStore().storeSummary(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
summaryForStore,
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens,
|
||||
originalTimestamp ?? undefined
|
||||
);
|
||||
|
||||
logger.info('SDK', 'Gemini summary saved', {
|
||||
sessionId: session.sessionDbId,
|
||||
summaryId,
|
||||
request: summary.request || '(no request)'
|
||||
});
|
||||
|
||||
// Sync to Chroma
|
||||
this.dbManager.getChromaSync().syncSummary(
|
||||
summaryId,
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
summaryForStore,
|
||||
session.lastPromptNumber,
|
||||
createdAtEpoch,
|
||||
discoveryTokens
|
||||
).catch(err => {
|
||||
logger.warn('SDK', 'Gemini chroma sync failed', { summaryId }, err);
|
||||
});
|
||||
|
||||
// Broadcast to SSE clients
|
||||
if (worker && worker.sseBroadcaster) {
|
||||
worker.sseBroadcaster.broadcast({
|
||||
type: 'new_summary',
|
||||
summary: {
|
||||
id: summaryId,
|
||||
session_id: session.claudeSessionId,
|
||||
request: summary.request,
|
||||
investigated: summary.investigated,
|
||||
learned: summary.learned,
|
||||
completed: summary.completed,
|
||||
next_steps: summary.next_steps,
|
||||
notes: summary.notes,
|
||||
project: session.project,
|
||||
prompt_number: session.lastPromptNumber,
|
||||
created_at_epoch: createdAtEpoch
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Mark messages as processed
|
||||
await this.markMessagesProcessed(session, worker);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark pending messages as processed
|
||||
*/
|
||||
private async markMessagesProcessed(session: ActiveSession, worker: any | undefined): Promise<void> {
|
||||
const pendingMessageStore = this.sessionManager.getPendingMessageStore();
|
||||
if (session.pendingProcessingIds.size > 0) {
|
||||
for (const messageId of session.pendingProcessingIds) {
|
||||
pendingMessageStore.markProcessed(messageId);
|
||||
}
|
||||
logger.debug('SDK', 'Gemini messages marked as processed', {
|
||||
sessionId: session.sessionDbId,
|
||||
count: session.pendingProcessingIds.size
|
||||
});
|
||||
session.pendingProcessingIds.clear();
|
||||
|
||||
const deletedCount = pendingMessageStore.cleanupProcessed(100);
|
||||
if (deletedCount > 0) {
|
||||
logger.debug('SDK', 'Gemini cleaned up old processed messages', { deletedCount });
|
||||
}
|
||||
}
|
||||
|
||||
if (worker && typeof worker.broadcastProcessingStatus === 'function') {
|
||||
worker.broadcastProcessingStatus();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get Gemini configuration from settings or environment
|
||||
*/
|
||||
private getGeminiConfig(): { apiKey: string; model: GeminiModel; rateLimitingEnabled: boolean } {
|
||||
const settingsPath = path.join(homedir(), '.claude-mem', 'settings.json');
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
|
||||
// API key: check settings first, then environment variable
|
||||
const apiKey = settings.CLAUDE_MEM_GEMINI_API_KEY || process.env.GEMINI_API_KEY || '';
|
||||
|
||||
// Model: from settings or default, with validation
|
||||
const defaultModel: GeminiModel = 'gemini-2.5-flash';
|
||||
const configuredModel = settings.CLAUDE_MEM_GEMINI_MODEL || defaultModel;
|
||||
const validModels: GeminiModel[] = [
|
||||
'gemini-2.5-flash-lite',
|
||||
'gemini-2.5-flash',
|
||||
'gemini-2.5-pro',
|
||||
'gemini-2.0-flash',
|
||||
'gemini-2.0-flash-lite',
|
||||
];
|
||||
|
||||
let model: GeminiModel;
|
||||
if (validModels.includes(configuredModel as GeminiModel)) {
|
||||
model = configuredModel as GeminiModel;
|
||||
} else {
|
||||
logger.warn('SDK', `Invalid Gemini model "${configuredModel}", falling back to ${defaultModel}`, {
|
||||
configured: configuredModel,
|
||||
validModels,
|
||||
});
|
||||
model = defaultModel;
|
||||
}
|
||||
|
||||
// Rate limiting: enabled by default for free tier users
|
||||
const rateLimitingEnabled = settings.CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED !== 'false';
|
||||
|
||||
return { apiKey, model, rateLimitingEnabled };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Gemini is available (has API key configured)
|
||||
*/
|
||||
export function isGeminiAvailable(): boolean {
|
||||
const settingsPath = path.join(homedir(), '.claude-mem', 'settings.json');
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
return !!(settings.CLAUDE_MEM_GEMINI_API_KEY || process.env.GEMINI_API_KEY);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Gemini is the selected provider
|
||||
*/
|
||||
export function isGeminiSelected(): boolean {
|
||||
const settingsPath = path.join(homedir(), '.claude-mem', 'settings.json');
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
return settings.CLAUDE_MEM_PROVIDER === 'gemini';
|
||||
}
|
||||
@@ -0,0 +1,608 @@
|
||||
/**
|
||||
* OpenRouterAgent: OpenRouter-based observation extraction
|
||||
*
|
||||
* Alternative to SDKAgent that uses OpenRouter's unified API
|
||||
* for accessing 100+ models from different providers.
|
||||
*
|
||||
* Responsibility:
|
||||
* - Call OpenRouter REST API for observation extraction
|
||||
* - Parse XML responses (same format as Claude/Gemini)
|
||||
* - Sync to database and Chroma
|
||||
* - Support dynamic model selection across providers
|
||||
*/
|
||||
|
||||
import { DatabaseManager } from './DatabaseManager.js';
|
||||
import { SessionManager } from './SessionManager.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { parseObservations, parseSummary } from '../../sdk/parser.js';
|
||||
import { buildInitPrompt, buildObservationPrompt, buildSummaryPrompt, buildContinuationPrompt } from '../../sdk/prompts.js';
|
||||
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
|
||||
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
|
||||
import type { ActiveSession, ConversationMessage } from '../worker-types.js';
|
||||
import { ModeManager } from '../domain/ModeManager.js';
|
||||
|
||||
// OpenRouter API endpoint
|
||||
const OPENROUTER_API_URL = 'https://openrouter.ai/api/v1/chat/completions';
|
||||
|
||||
// Context window management constants (defaults, overridable via settings)
|
||||
const DEFAULT_MAX_CONTEXT_MESSAGES = 20; // Maximum messages to keep in conversation history
|
||||
const DEFAULT_MAX_ESTIMATED_TOKENS = 100000; // ~100k tokens max context (safety limit)
|
||||
const CHARS_PER_TOKEN_ESTIMATE = 4; // Conservative estimate: 1 token ≈ 4 chars
|
||||
|
||||
// OpenAI-compatible message format
|
||||
interface OpenAIMessage {
|
||||
role: 'user' | 'assistant' | 'system';
|
||||
content: string;
|
||||
}
|
||||
|
||||
interface OpenRouterResponse {
|
||||
choices?: Array<{
|
||||
message?: {
|
||||
role?: string;
|
||||
content?: string;
|
||||
};
|
||||
finish_reason?: string;
|
||||
}>;
|
||||
usage?: {
|
||||
prompt_tokens?: number;
|
||||
completion_tokens?: number;
|
||||
total_tokens?: number;
|
||||
};
|
||||
error?: {
|
||||
message?: string;
|
||||
code?: string;
|
||||
};
|
||||
}
|
||||
|
||||
// Forward declaration for fallback agent type
|
||||
type FallbackAgent = {
|
||||
startSession(session: ActiveSession, worker?: any): Promise<void>;
|
||||
};
|
||||
|
||||
export class OpenRouterAgent {
|
||||
private dbManager: DatabaseManager;
|
||||
private sessionManager: SessionManager;
|
||||
private fallbackAgent: FallbackAgent | null = null;
|
||||
|
||||
constructor(dbManager: DatabaseManager, sessionManager: SessionManager) {
|
||||
this.dbManager = dbManager;
|
||||
this.sessionManager = sessionManager;
|
||||
}
|
||||
|
||||
/**
|
||||
* Set the fallback agent (Claude SDK) for when OpenRouter API fails
|
||||
* Must be set after construction to avoid circular dependency
|
||||
*/
|
||||
setFallbackAgent(agent: FallbackAgent): void {
|
||||
this.fallbackAgent = agent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if an error should trigger fallback to Claude
|
||||
*/
|
||||
private shouldFallbackToClaude(error: any): boolean {
|
||||
const message = error?.message || '';
|
||||
// Fall back on rate limit (429), server errors (5xx), or network issues
|
||||
return (
|
||||
message.includes('429') ||
|
||||
message.includes('500') ||
|
||||
message.includes('502') ||
|
||||
message.includes('503') ||
|
||||
message.includes('ECONNREFUSED') ||
|
||||
message.includes('ETIMEDOUT') ||
|
||||
message.includes('fetch failed')
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Start OpenRouter agent for a session
|
||||
* Uses multi-turn conversation to maintain context across messages
|
||||
*/
|
||||
async startSession(session: ActiveSession, worker?: any): Promise<void> {
|
||||
try {
|
||||
// Get OpenRouter configuration
|
||||
const { apiKey, model, siteUrl, appName } = this.getOpenRouterConfig();
|
||||
|
||||
if (!apiKey) {
|
||||
throw new Error('OpenRouter API key not configured. Set CLAUDE_MEM_OPENROUTER_API_KEY in settings or OPENROUTER_API_KEY environment variable.');
|
||||
}
|
||||
|
||||
// Load active mode
|
||||
const mode = ModeManager.getInstance().getActiveMode();
|
||||
|
||||
// Build initial prompt
|
||||
const initPrompt = session.lastPromptNumber === 1
|
||||
? buildInitPrompt(session.project, session.claudeSessionId, session.userPrompt, mode)
|
||||
: buildContinuationPrompt(session.userPrompt, session.lastPromptNumber, session.claudeSessionId, mode);
|
||||
|
||||
// Add to conversation history and query OpenRouter with full context
|
||||
session.conversationHistory.push({ role: 'user', content: initPrompt });
|
||||
const initResponse = await this.queryOpenRouterMultiTurn(session.conversationHistory, apiKey, model, siteUrl, appName);
|
||||
|
||||
if (initResponse.content) {
|
||||
// Add response to conversation history
|
||||
session.conversationHistory.push({ role: 'assistant', content: initResponse.content });
|
||||
|
||||
// Track token usage
|
||||
const tokensUsed = initResponse.tokensUsed || 0;
|
||||
session.cumulativeInputTokens += Math.floor(tokensUsed * 0.7); // Rough estimate
|
||||
session.cumulativeOutputTokens += Math.floor(tokensUsed * 0.3);
|
||||
|
||||
// Process response (no original timestamp for init - not from queue)
|
||||
await this.processOpenRouterResponse(session, initResponse.content, worker, tokensUsed, null);
|
||||
} else {
|
||||
logger.warn('SDK', 'Empty OpenRouter init response - session may lack context', {
|
||||
sessionId: session.sessionDbId,
|
||||
model
|
||||
});
|
||||
}
|
||||
|
||||
// Process pending messages
|
||||
for await (const message of this.sessionManager.getMessageIterator(session.sessionDbId)) {
|
||||
// Capture earliest timestamp BEFORE processing (will be cleared after)
|
||||
const originalTimestamp = session.earliestPendingTimestamp;
|
||||
|
||||
if (message.type === 'observation') {
|
||||
// Update last prompt number
|
||||
if (message.prompt_number !== undefined) {
|
||||
session.lastPromptNumber = message.prompt_number;
|
||||
}
|
||||
|
||||
// Build observation prompt
|
||||
const obsPrompt = buildObservationPrompt({
|
||||
id: 0,
|
||||
tool_name: message.tool_name!,
|
||||
tool_input: JSON.stringify(message.tool_input),
|
||||
tool_output: JSON.stringify(message.tool_response),
|
||||
created_at_epoch: originalTimestamp ?? Date.now(),
|
||||
cwd: message.cwd
|
||||
});
|
||||
|
||||
// Add to conversation history and query OpenRouter with full context
|
||||
session.conversationHistory.push({ role: 'user', content: obsPrompt });
|
||||
const obsResponse = await this.queryOpenRouterMultiTurn(session.conversationHistory, apiKey, model, siteUrl, appName);
|
||||
|
||||
if (obsResponse.content) {
|
||||
// Add response to conversation history
|
||||
session.conversationHistory.push({ role: 'assistant', content: obsResponse.content });
|
||||
|
||||
const tokensUsed = obsResponse.tokensUsed || 0;
|
||||
session.cumulativeInputTokens += Math.floor(tokensUsed * 0.7);
|
||||
session.cumulativeOutputTokens += Math.floor(tokensUsed * 0.3);
|
||||
await this.processOpenRouterResponse(session, obsResponse.content, worker, tokensUsed, originalTimestamp);
|
||||
} else {
|
||||
// Empty response - still mark messages as processed to avoid stuck state
|
||||
logger.warn('SDK', 'Empty OpenRouter response for observation, marking as processed', {
|
||||
sessionId: session.sessionDbId,
|
||||
toolName: message.tool_name
|
||||
});
|
||||
await this.markMessagesProcessed(session, worker);
|
||||
}
|
||||
|
||||
} else if (message.type === 'summarize') {
|
||||
// Build summary prompt
|
||||
const summaryPrompt = buildSummaryPrompt({
|
||||
id: session.sessionDbId,
|
||||
sdk_session_id: session.sdkSessionId,
|
||||
project: session.project,
|
||||
user_prompt: session.userPrompt,
|
||||
last_user_message: message.last_user_message || '',
|
||||
last_assistant_message: message.last_assistant_message || ''
|
||||
}, mode);
|
||||
|
||||
// Add to conversation history and query OpenRouter with full context
|
||||
session.conversationHistory.push({ role: 'user', content: summaryPrompt });
|
||||
const summaryResponse = await this.queryOpenRouterMultiTurn(session.conversationHistory, apiKey, model, siteUrl, appName);
|
||||
|
||||
if (summaryResponse.content) {
|
||||
// Add response to conversation history
|
||||
session.conversationHistory.push({ role: 'assistant', content: summaryResponse.content });
|
||||
|
||||
const tokensUsed = summaryResponse.tokensUsed || 0;
|
||||
session.cumulativeInputTokens += Math.floor(tokensUsed * 0.7);
|
||||
session.cumulativeOutputTokens += Math.floor(tokensUsed * 0.3);
|
||||
await this.processOpenRouterResponse(session, summaryResponse.content, worker, tokensUsed, originalTimestamp);
|
||||
} else {
|
||||
// Empty response - still mark messages as processed to avoid stuck state
|
||||
logger.warn('SDK', 'Empty OpenRouter response for summary, marking as processed', {
|
||||
sessionId: session.sessionDbId
|
||||
});
|
||||
await this.markMessagesProcessed(session, worker);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Mark session complete
|
||||
const sessionDuration = Date.now() - session.startTime;
|
||||
logger.success('SDK', 'OpenRouter agent completed', {
|
||||
sessionId: session.sessionDbId,
|
||||
duration: `${(sessionDuration / 1000).toFixed(1)}s`,
|
||||
historyLength: session.conversationHistory.length,
|
||||
model
|
||||
});
|
||||
|
||||
} catch (error: any) {
|
||||
if (error.name === 'AbortError') {
|
||||
logger.warn('SDK', 'OpenRouter agent aborted', { sessionId: session.sessionDbId });
|
||||
throw error;
|
||||
}
|
||||
|
||||
// Check if we should fall back to Claude
|
||||
if (this.shouldFallbackToClaude(error) && this.fallbackAgent) {
|
||||
logger.warn('SDK', 'OpenRouter API failed, falling back to Claude SDK', {
|
||||
sessionDbId: session.sessionDbId,
|
||||
error: error.message,
|
||||
historyLength: session.conversationHistory.length
|
||||
});
|
||||
|
||||
// Reset any 'processing' messages back to 'pending' so Claude can retry them
|
||||
const pendingStore = this.sessionManager.getPendingMessageStore();
|
||||
const resetCount = pendingStore.resetStuckMessages(0); // 0 = reset ALL processing messages
|
||||
if (resetCount > 0) {
|
||||
logger.info('SDK', 'Reset processing messages for fallback', {
|
||||
sessionDbId: session.sessionDbId,
|
||||
resetCount
|
||||
});
|
||||
}
|
||||
|
||||
// Fall back to Claude - it will use the same session with shared conversationHistory
|
||||
return this.fallbackAgent.startSession(session, worker);
|
||||
}
|
||||
|
||||
logger.failure('SDK', 'OpenRouter agent error', { sessionDbId: session.sessionDbId }, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Estimate token count from text (conservative estimate)
|
||||
*/
|
||||
private estimateTokens(text: string): number {
|
||||
return Math.ceil(text.length / CHARS_PER_TOKEN_ESTIMATE);
|
||||
}
|
||||
|
||||
/**
|
||||
* Truncate conversation history to prevent runaway context costs
|
||||
* Keeps most recent messages within token budget
|
||||
*/
|
||||
private truncateHistory(history: ConversationMessage[]): ConversationMessage[] {
|
||||
const settings = SettingsDefaultsManager.loadFromFile(
|
||||
USER_SETTINGS_PATH
|
||||
);
|
||||
|
||||
const MAX_CONTEXT_MESSAGES = parseInt(settings.CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES) || DEFAULT_MAX_CONTEXT_MESSAGES;
|
||||
const MAX_ESTIMATED_TOKENS = parseInt(settings.CLAUDE_MEM_OPENROUTER_MAX_TOKENS) || DEFAULT_MAX_ESTIMATED_TOKENS;
|
||||
|
||||
if (history.length <= MAX_CONTEXT_MESSAGES) {
|
||||
// Check token count even if message count is ok
|
||||
const totalTokens = history.reduce((sum, m) => sum + this.estimateTokens(m.content), 0);
|
||||
if (totalTokens <= MAX_ESTIMATED_TOKENS) {
|
||||
return history;
|
||||
}
|
||||
}
|
||||
|
||||
// Sliding window: keep most recent messages within limits
|
||||
const truncated: ConversationMessage[] = [];
|
||||
let tokenCount = 0;
|
||||
|
||||
// Process messages in reverse (most recent first)
|
||||
for (let i = history.length - 1; i >= 0; i--) {
|
||||
const msg = history[i];
|
||||
const msgTokens = this.estimateTokens(msg.content);
|
||||
|
||||
if (truncated.length >= MAX_CONTEXT_MESSAGES || tokenCount + msgTokens > MAX_ESTIMATED_TOKENS) {
|
||||
logger.warn('SDK', 'Context window truncated to prevent runaway costs', {
|
||||
originalMessages: history.length,
|
||||
keptMessages: truncated.length,
|
||||
droppedMessages: i + 1,
|
||||
estimatedTokens: tokenCount,
|
||||
tokenLimit: MAX_ESTIMATED_TOKENS
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
truncated.unshift(msg); // Add to beginning
|
||||
tokenCount += msgTokens;
|
||||
}
|
||||
|
||||
return truncated;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert shared ConversationMessage array to OpenAI-compatible message format
|
||||
*/
|
||||
private conversationToOpenAIMessages(history: ConversationMessage[]): OpenAIMessage[] {
|
||||
return history.map(msg => ({
|
||||
role: msg.role === 'assistant' ? 'assistant' : 'user',
|
||||
content: msg.content
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Query OpenRouter via REST API with full conversation history (multi-turn)
|
||||
* Sends the entire conversation context for coherent responses
|
||||
*/
|
||||
private async queryOpenRouterMultiTurn(
|
||||
history: ConversationMessage[],
|
||||
apiKey: string,
|
||||
model: string,
|
||||
siteUrl?: string,
|
||||
appName?: string
|
||||
): Promise<{ content: string; tokensUsed?: number }> {
|
||||
// Truncate history to prevent runaway costs
|
||||
const truncatedHistory = this.truncateHistory(history);
|
||||
const messages = this.conversationToOpenAIMessages(truncatedHistory);
|
||||
const totalChars = truncatedHistory.reduce((sum, m) => sum + m.content.length, 0);
|
||||
const estimatedTokens = this.estimateTokens(truncatedHistory.map(m => m.content).join(''));
|
||||
|
||||
logger.debug('SDK', `Querying OpenRouter multi-turn (${model})`, {
|
||||
turns: truncatedHistory.length,
|
||||
totalChars,
|
||||
estimatedTokens
|
||||
});
|
||||
|
||||
const response = await fetch(OPENROUTER_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'HTTP-Referer': siteUrl || 'https://github.com/thedotmack/claude-mem',
|
||||
'X-Title': appName || 'claude-mem',
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model,
|
||||
messages,
|
||||
temperature: 0.3, // Lower temperature for structured extraction
|
||||
max_tokens: 4096,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
throw new Error(`OpenRouter API error: ${response.status} - ${errorText}`);
|
||||
}
|
||||
|
||||
const data = await response.json() as OpenRouterResponse;
|
||||
|
||||
// Check for API error in response body
|
||||
if (data.error) {
|
||||
throw new Error(`OpenRouter API error: ${data.error.code} - ${data.error.message}`);
|
||||
}
|
||||
|
||||
if (!data.choices?.[0]?.message?.content) {
|
||||
logger.warn('SDK', 'Empty response from OpenRouter');
|
||||
return { content: '' };
|
||||
}
|
||||
|
||||
const content = data.choices[0].message.content;
|
||||
const tokensUsed = data.usage?.total_tokens;
|
||||
|
||||
// Log actual token usage for cost tracking
|
||||
if (tokensUsed) {
|
||||
const inputTokens = data.usage?.prompt_tokens || 0;
|
||||
const outputTokens = data.usage?.completion_tokens || 0;
|
||||
// Token usage (cost varies by model - many OpenRouter models are free)
|
||||
const estimatedCost = (inputTokens / 1000000 * 3) + (outputTokens / 1000000 * 15);
|
||||
|
||||
logger.info('SDK', 'OpenRouter API usage', {
|
||||
model,
|
||||
inputTokens,
|
||||
outputTokens,
|
||||
totalTokens: tokensUsed,
|
||||
estimatedCostUSD: estimatedCost.toFixed(4),
|
||||
messagesInContext: truncatedHistory.length
|
||||
});
|
||||
|
||||
// Warn if costs are getting high
|
||||
if (tokensUsed > 50000) {
|
||||
logger.warn('SDK', 'High token usage detected - consider reducing context', {
|
||||
totalTokens: tokensUsed,
|
||||
estimatedCost: estimatedCost.toFixed(4)
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return { content, tokensUsed };
|
||||
}
|
||||
|
||||
/**
|
||||
* Process OpenRouter response (same format as Claude/Gemini)
|
||||
* @param originalTimestamp - Original epoch when message was queued (for backlog processing accuracy)
|
||||
*/
|
||||
private async processOpenRouterResponse(
|
||||
session: ActiveSession,
|
||||
text: string,
|
||||
worker: any | undefined,
|
||||
discoveryTokens: number,
|
||||
originalTimestamp: number | null
|
||||
): Promise<void> {
|
||||
// Parse observations (same XML format)
|
||||
const observations = parseObservations(text, session.claudeSessionId);
|
||||
|
||||
// Store observations with original timestamp (if processing backlog) or current time
|
||||
for (const obs of observations) {
|
||||
const { id: obsId, createdAtEpoch } = this.dbManager.getSessionStore().storeObservation(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens,
|
||||
originalTimestamp ?? undefined
|
||||
);
|
||||
|
||||
logger.info('SDK', 'OpenRouter observation saved', {
|
||||
sessionId: session.sessionDbId,
|
||||
obsId,
|
||||
type: obs.type,
|
||||
title: obs.title || '(untitled)'
|
||||
});
|
||||
|
||||
// Sync to Chroma
|
||||
this.dbManager.getChromaSync().syncObservation(
|
||||
obsId,
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
createdAtEpoch,
|
||||
discoveryTokens
|
||||
).catch(err => {
|
||||
logger.warn('SDK', 'OpenRouter chroma sync failed', { obsId }, err);
|
||||
});
|
||||
|
||||
// Broadcast to SSE clients
|
||||
if (worker && worker.sseBroadcaster) {
|
||||
worker.sseBroadcaster.broadcast({
|
||||
type: 'new_observation',
|
||||
observation: {
|
||||
id: obsId,
|
||||
sdk_session_id: session.sdkSessionId,
|
||||
session_id: session.claudeSessionId,
|
||||
type: obs.type,
|
||||
title: obs.title,
|
||||
subtitle: obs.subtitle,
|
||||
text: null,
|
||||
narrative: obs.narrative || null,
|
||||
facts: JSON.stringify(obs.facts || []),
|
||||
concepts: JSON.stringify(obs.concepts || []),
|
||||
files_read: JSON.stringify(obs.files_read || []),
|
||||
files_modified: JSON.stringify(obs.files_modified || []),
|
||||
project: session.project,
|
||||
prompt_number: session.lastPromptNumber,
|
||||
created_at_epoch: createdAtEpoch
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Parse summary
|
||||
const summary = parseSummary(text, session.sessionDbId);
|
||||
|
||||
if (summary) {
|
||||
// Convert nullable fields to empty strings for storeSummary
|
||||
const summaryForStore = {
|
||||
request: summary.request || '',
|
||||
investigated: summary.investigated || '',
|
||||
learned: summary.learned || '',
|
||||
completed: summary.completed || '',
|
||||
next_steps: summary.next_steps || '',
|
||||
notes: summary.notes
|
||||
};
|
||||
|
||||
const { id: summaryId, createdAtEpoch } = this.dbManager.getSessionStore().storeSummary(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
summaryForStore,
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens,
|
||||
originalTimestamp ?? undefined
|
||||
);
|
||||
|
||||
logger.info('SDK', 'OpenRouter summary saved', {
|
||||
sessionId: session.sessionDbId,
|
||||
summaryId,
|
||||
request: summary.request || '(no request)'
|
||||
});
|
||||
|
||||
// Sync to Chroma
|
||||
this.dbManager.getChromaSync().syncSummary(
|
||||
summaryId,
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
summaryForStore,
|
||||
session.lastPromptNumber,
|
||||
createdAtEpoch,
|
||||
discoveryTokens
|
||||
).catch(err => {
|
||||
logger.warn('SDK', 'OpenRouter chroma sync failed', { summaryId }, err);
|
||||
});
|
||||
|
||||
// Broadcast to SSE clients
|
||||
if (worker && worker.sseBroadcaster) {
|
||||
worker.sseBroadcaster.broadcast({
|
||||
type: 'new_summary',
|
||||
summary: {
|
||||
id: summaryId,
|
||||
session_id: session.claudeSessionId,
|
||||
request: summary.request,
|
||||
investigated: summary.investigated,
|
||||
learned: summary.learned,
|
||||
completed: summary.completed,
|
||||
next_steps: summary.next_steps,
|
||||
notes: summary.notes,
|
||||
project: session.project,
|
||||
prompt_number: session.lastPromptNumber,
|
||||
created_at_epoch: createdAtEpoch
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Mark messages as processed
|
||||
await this.markMessagesProcessed(session, worker);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark pending messages as processed
|
||||
*/
|
||||
private async markMessagesProcessed(session: ActiveSession, worker: any | undefined): Promise<void> {
|
||||
const pendingMessageStore = this.sessionManager.getPendingMessageStore();
|
||||
if (session.pendingProcessingIds.size > 0) {
|
||||
for (const messageId of session.pendingProcessingIds) {
|
||||
pendingMessageStore.markProcessed(messageId);
|
||||
}
|
||||
logger.debug('SDK', 'OpenRouter messages marked as processed', {
|
||||
sessionId: session.sessionDbId,
|
||||
count: session.pendingProcessingIds.size
|
||||
});
|
||||
session.pendingProcessingIds.clear();
|
||||
|
||||
const deletedCount = pendingMessageStore.cleanupProcessed(100);
|
||||
if (deletedCount > 0) {
|
||||
logger.debug('SDK', 'OpenRouter cleaned up old processed messages', { deletedCount });
|
||||
}
|
||||
}
|
||||
|
||||
if (worker && typeof worker.broadcastProcessingStatus === 'function') {
|
||||
worker.broadcastProcessingStatus();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get OpenRouter configuration from settings or environment
|
||||
*/
|
||||
private getOpenRouterConfig(): { apiKey: string; model: string; siteUrl?: string; appName?: string } {
|
||||
const settingsPath = USER_SETTINGS_PATH;
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
|
||||
// API key: check settings first, then environment variable
|
||||
const apiKey = settings.CLAUDE_MEM_OPENROUTER_API_KEY || process.env.OPENROUTER_API_KEY || '';
|
||||
|
||||
// Model: from settings or default
|
||||
const model = settings.CLAUDE_MEM_OPENROUTER_MODEL || 'xiaomi/mimo-v2-flash:free';
|
||||
|
||||
// Optional analytics headers
|
||||
const siteUrl = settings.CLAUDE_MEM_OPENROUTER_SITE_URL || '';
|
||||
const appName = settings.CLAUDE_MEM_OPENROUTER_APP_NAME || 'claude-mem';
|
||||
|
||||
return { apiKey, model, siteUrl, appName };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if OpenRouter is available (has API key configured)
|
||||
*/
|
||||
export function isOpenRouterAvailable(): boolean {
|
||||
const settingsPath = USER_SETTINGS_PATH;
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
return !!(settings.CLAUDE_MEM_OPENROUTER_API_KEY || process.env.OPENROUTER_API_KEY);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if OpenRouter is the selected provider
|
||||
*/
|
||||
export function isOpenRouterSelected(): boolean {
|
||||
const settingsPath = USER_SETTINGS_PATH;
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
return settings.CLAUDE_MEM_PROVIDER === 'openrouter';
|
||||
}
|
||||
@@ -8,6 +8,7 @@
|
||||
*/
|
||||
|
||||
import { DatabaseManager } from './DatabaseManager.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import type { PaginatedResult, Observation, Summary, UserPrompt } from '../worker-types.js';
|
||||
|
||||
export class PaginationHelper {
|
||||
|
||||
+114
-35
@@ -64,11 +64,19 @@ export class SDKAgent {
|
||||
// Create message generator (event-driven)
|
||||
const messageGenerator = this.createMessageGenerator(session);
|
||||
|
||||
logger.info('SDK', 'Starting SDK query', {
|
||||
sessionDbId: session.sessionDbId,
|
||||
claudeSessionId: session.claudeSessionId,
|
||||
resume_parameter: session.claudeSessionId,
|
||||
lastPromptNumber: session.lastPromptNumber
|
||||
});
|
||||
|
||||
// Run Agent SDK query loop
|
||||
const queryResult = query({
|
||||
prompt: messageGenerator,
|
||||
options: {
|
||||
model: modelId,
|
||||
resume: session.claudeSessionId,
|
||||
disallowedTools,
|
||||
abortController: session.abortController,
|
||||
pathToClaudeCodeExecutable: claudePath
|
||||
@@ -115,6 +123,9 @@ export class SDKAgent {
|
||||
const discoveryTokens = (session.cumulativeInputTokens + session.cumulativeOutputTokens) - tokensBeforeResponse;
|
||||
|
||||
// Process response (empty or not) and mark messages as processed
|
||||
// Capture earliest timestamp BEFORE processing (will be cleared after)
|
||||
const originalTimestamp = session.earliestPendingTimestamp;
|
||||
|
||||
if (responseSize > 0) {
|
||||
const truncatedResponse = responseSize > 100
|
||||
? textContent.substring(0, 100) + '...'
|
||||
@@ -124,8 +135,8 @@ export class SDKAgent {
|
||||
promptNumber: session.lastPromptNumber
|
||||
}, truncatedResponse);
|
||||
|
||||
// Parse and process response with discovery token delta
|
||||
await this.processSDKResponse(session, textContent, worker, discoveryTokens);
|
||||
// Parse and process response with discovery token delta and original timestamp
|
||||
await this.processSDKResponse(session, textContent, worker, discoveryTokens, originalTimestamp);
|
||||
} else {
|
||||
// Empty response - still need to mark pending messages as processed
|
||||
await this.markMessagesProcessed(session, worker);
|
||||
@@ -145,8 +156,6 @@ export class SDKAgent {
|
||||
duration: `${(sessionDuration / 1000).toFixed(1)}s`
|
||||
});
|
||||
|
||||
this.dbManager.getSessionStore().markSessionCompleted(session.sessionDbId);
|
||||
|
||||
} catch (error: any) {
|
||||
if (error.name === 'AbortError') {
|
||||
logger.warn('SDK', 'Agent aborted', { sessionId: session.sessionDbId });
|
||||
@@ -184,20 +193,40 @@ export class SDKAgent {
|
||||
* - SessionManager.initializeSession already fetched this from database
|
||||
* - Database row was created by new-hook's createSDKSession call
|
||||
* - We just use the session_id we're given - simple and reliable
|
||||
*
|
||||
* SHARED CONVERSATION HISTORY:
|
||||
* - Each user message is added to session.conversationHistory
|
||||
* - This allows provider switching (Claude→Gemini) with full context
|
||||
* - SDK manages its own internal state, but we mirror it for interop
|
||||
*/
|
||||
private async *createMessageGenerator(session: ActiveSession): AsyncIterableIterator<SDKUserMessage> {
|
||||
// Load active mode
|
||||
const mode = ModeManager.getInstance().getActiveMode();
|
||||
|
||||
// Build initial prompt
|
||||
const isInitPrompt = session.lastPromptNumber === 1;
|
||||
logger.info('SDK', 'Creating message generator', {
|
||||
sessionDbId: session.sessionDbId,
|
||||
claudeSessionId: session.claudeSessionId,
|
||||
lastPromptNumber: session.lastPromptNumber,
|
||||
isInitPrompt,
|
||||
promptType: isInitPrompt ? 'INIT' : 'CONTINUATION'
|
||||
});
|
||||
|
||||
const initPrompt = isInitPrompt
|
||||
? buildInitPrompt(session.project, session.claudeSessionId, session.userPrompt, mode)
|
||||
: buildContinuationPrompt(session.userPrompt, session.lastPromptNumber, session.claudeSessionId, mode);
|
||||
|
||||
// Add to shared conversation history for provider interop
|
||||
session.conversationHistory.push({ role: 'user', content: initPrompt });
|
||||
|
||||
// Yield initial user prompt with context (or continuation if prompt #2+)
|
||||
// CRITICAL: Both paths use session.claudeSessionId from the hook
|
||||
yield {
|
||||
type: 'user',
|
||||
message: {
|
||||
role: 'user',
|
||||
content: session.lastPromptNumber === 1
|
||||
? buildInitPrompt(session.project, session.claudeSessionId, session.userPrompt, mode)
|
||||
: buildContinuationPrompt(session.userPrompt, session.lastPromptNumber, session.claudeSessionId, mode)
|
||||
content: initPrompt
|
||||
},
|
||||
session_id: session.claudeSessionId,
|
||||
parent_tool_use_id: null,
|
||||
@@ -212,36 +241,46 @@ export class SDKAgent {
|
||||
session.lastPromptNumber = message.prompt_number;
|
||||
}
|
||||
|
||||
const obsPrompt = buildObservationPrompt({
|
||||
id: 0, // Not used in prompt
|
||||
tool_name: message.tool_name!,
|
||||
tool_input: JSON.stringify(message.tool_input),
|
||||
tool_output: JSON.stringify(message.tool_response),
|
||||
created_at_epoch: Date.now(),
|
||||
cwd: message.cwd
|
||||
});
|
||||
|
||||
// Add to shared conversation history for provider interop
|
||||
session.conversationHistory.push({ role: 'user', content: obsPrompt });
|
||||
|
||||
yield {
|
||||
type: 'user',
|
||||
message: {
|
||||
role: 'user',
|
||||
content: buildObservationPrompt({
|
||||
id: 0, // Not used in prompt
|
||||
tool_name: message.tool_name!,
|
||||
tool_input: JSON.stringify(message.tool_input),
|
||||
tool_output: JSON.stringify(message.tool_response),
|
||||
created_at_epoch: Date.now(),
|
||||
cwd: message.cwd
|
||||
})
|
||||
content: obsPrompt
|
||||
},
|
||||
session_id: session.claudeSessionId,
|
||||
parent_tool_use_id: null,
|
||||
isSynthetic: true
|
||||
};
|
||||
} else if (message.type === 'summarize') {
|
||||
const summaryPrompt = buildSummaryPrompt({
|
||||
id: session.sessionDbId,
|
||||
sdk_session_id: session.sdkSessionId,
|
||||
project: session.project,
|
||||
user_prompt: session.userPrompt,
|
||||
last_user_message: message.last_user_message || '',
|
||||
last_assistant_message: message.last_assistant_message || ''
|
||||
}, mode);
|
||||
|
||||
// Add to shared conversation history for provider interop
|
||||
session.conversationHistory.push({ role: 'user', content: summaryPrompt });
|
||||
|
||||
yield {
|
||||
type: 'user',
|
||||
message: {
|
||||
role: 'user',
|
||||
content: buildSummaryPrompt({
|
||||
id: session.sessionDbId,
|
||||
sdk_session_id: session.sdkSessionId,
|
||||
project: session.project,
|
||||
user_prompt: session.userPrompt,
|
||||
last_user_message: message.last_user_message || '',
|
||||
last_assistant_message: message.last_assistant_message || ''
|
||||
}, mode)
|
||||
content: summaryPrompt
|
||||
},
|
||||
session_id: session.claudeSessionId,
|
||||
parent_tool_use_id: null,
|
||||
@@ -254,19 +293,29 @@ export class SDKAgent {
|
||||
/**
|
||||
* Process SDK response text (parse XML, save to database, sync to Chroma)
|
||||
* @param discoveryTokens - Token cost for discovering this response (delta, not cumulative)
|
||||
* @param originalTimestamp - Original epoch when message was queued (for backlog processing accuracy)
|
||||
*
|
||||
* Also captures assistant responses to shared conversation history for provider interop.
|
||||
* This allows Gemini to see full context if provider is switched mid-session.
|
||||
*/
|
||||
private async processSDKResponse(session: ActiveSession, text: string, worker: any | undefined, discoveryTokens: number): Promise<void> {
|
||||
private async processSDKResponse(session: ActiveSession, text: string, worker: any | undefined, discoveryTokens: number, originalTimestamp: number | null): Promise<void> {
|
||||
// Add assistant response to shared conversation history for provider interop
|
||||
if (text) {
|
||||
session.conversationHistory.push({ role: 'assistant', content: text });
|
||||
}
|
||||
|
||||
// Parse observations
|
||||
const observations = parseObservations(text, session.claudeSessionId);
|
||||
|
||||
// Store observations
|
||||
// Store observations with original timestamp (if processing backlog) or current time
|
||||
for (const obs of observations) {
|
||||
const { id: obsId, createdAtEpoch } = this.dbManager.getSessionStore().storeObservation(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens
|
||||
discoveryTokens,
|
||||
originalTimestamp ?? undefined
|
||||
);
|
||||
|
||||
// Log observation details
|
||||
@@ -300,6 +349,12 @@ export class SDKAgent {
|
||||
type: obsType,
|
||||
title: obsTitle
|
||||
});
|
||||
}).catch((error) => {
|
||||
logger.warn('CHROMA', 'Observation sync failed, continuing without vector search', {
|
||||
obsId,
|
||||
type: obsType,
|
||||
title: obsTitle
|
||||
}, error);
|
||||
});
|
||||
|
||||
// Broadcast to SSE clients (for web UI)
|
||||
@@ -330,14 +385,15 @@ export class SDKAgent {
|
||||
// Parse summary
|
||||
const summary = parseSummary(text, session.sessionDbId);
|
||||
|
||||
// Store summary
|
||||
// Store summary with original timestamp (if processing backlog) or current time
|
||||
if (summary) {
|
||||
const { id: summaryId, createdAtEpoch } = this.dbManager.getSessionStore().storeSummary(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
summary,
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens
|
||||
discoveryTokens,
|
||||
originalTimestamp ?? undefined
|
||||
);
|
||||
|
||||
// Log summary details
|
||||
@@ -367,6 +423,11 @@ export class SDKAgent {
|
||||
duration: `${chromaDuration}ms`,
|
||||
request: summaryRequest
|
||||
});
|
||||
}).catch((error) => {
|
||||
logger.warn('CHROMA', 'Summary sync failed, continuing without vector search', {
|
||||
summaryId,
|
||||
request: summaryRequest
|
||||
}, error);
|
||||
});
|
||||
|
||||
// Broadcast to SSE clients (for web UI)
|
||||
@@ -411,6 +472,9 @@ export class SDKAgent {
|
||||
});
|
||||
session.pendingProcessingIds.clear();
|
||||
|
||||
// Clear timestamp for next batch (will be set fresh from next message)
|
||||
session.earliestPendingTimestamp = null;
|
||||
|
||||
// Clean up old processed messages (keep last 100 for UI display)
|
||||
const deletedCount = pendingMessageStore.cleanupProcessed(100);
|
||||
if (deletedCount > 0) {
|
||||
@@ -435,15 +499,30 @@ export class SDKAgent {
|
||||
*/
|
||||
private findClaudeExecutable(): string {
|
||||
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
|
||||
const claudePath = settings.CLAUDE_CODE_PATH ||
|
||||
execSync(process.platform === 'win32' ? 'where claude' : 'which claude', { encoding: 'utf8', windowsHide: true })
|
||||
.trim().split('\n')[0].trim();
|
||||
|
||||
if (!claudePath) {
|
||||
throw new Error('Claude executable not found in PATH');
|
||||
|
||||
// 1. Check configured path
|
||||
if (settings.CLAUDE_CODE_PATH) {
|
||||
// Lazy load fs to keep startup fast
|
||||
const { existsSync } = require('fs');
|
||||
if (!existsSync(settings.CLAUDE_CODE_PATH)) {
|
||||
throw new Error(`CLAUDE_CODE_PATH is set to "${settings.CLAUDE_CODE_PATH}" but the file does not exist.`);
|
||||
}
|
||||
return settings.CLAUDE_CODE_PATH;
|
||||
}
|
||||
|
||||
return claudePath;
|
||||
// 2. Try auto-detection
|
||||
try {
|
||||
const claudePath = execSync(
|
||||
process.platform === 'win32' ? 'where claude' : 'which claude',
|
||||
{ encoding: 'utf8', windowsHide: true, stdio: ['ignore', 'pipe', 'ignore'] }
|
||||
).trim().split('\n')[0].trim();
|
||||
|
||||
if (claudePath) return claudePath;
|
||||
} catch (error) {
|
||||
logger.debug('SDK', 'Claude executable auto-detection failed', error);
|
||||
}
|
||||
|
||||
throw new Error('Claude executable not found. Please either:\n1. Add "claude" to your system PATH, or\n2. Set CLAUDE_CODE_PATH in ~/.claude-mem/settings.json');
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -47,9 +47,21 @@ export class SessionManager {
|
||||
* Initialize a new session or return existing one
|
||||
*/
|
||||
initializeSession(sessionDbId: number, currentUserPrompt?: string, promptNumber?: number): ActiveSession {
|
||||
logger.info('SESSION', 'initializeSession called', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
has_currentUserPrompt: !!currentUserPrompt
|
||||
});
|
||||
|
||||
// Check if already active
|
||||
let session = this.sessions.get(sessionDbId);
|
||||
if (session) {
|
||||
logger.info('SESSION', 'Returning cached session', {
|
||||
sessionDbId,
|
||||
claudeSessionId: session.claudeSessionId,
|
||||
lastPromptNumber: session.lastPromptNumber
|
||||
});
|
||||
|
||||
// Refresh project from database in case it was updated by new-hook
|
||||
// This fixes the bug where sessions created with empty project get updated
|
||||
// in the database but the in-memory session still has the stale empty value
|
||||
@@ -86,6 +98,12 @@ export class SessionManager {
|
||||
// Fetch from database
|
||||
const dbSession = this.dbManager.getSessionById(sessionDbId);
|
||||
|
||||
logger.info('SESSION', 'Fetched session from database', {
|
||||
sessionDbId,
|
||||
claude_session_id: dbSession.claude_session_id,
|
||||
sdk_session_id: dbSession.sdk_session_id
|
||||
});
|
||||
|
||||
// Use currentUserPrompt if provided, otherwise fall back to database (first prompt)
|
||||
const userPrompt = currentUserPrompt || dbSession.user_prompt;
|
||||
|
||||
@@ -113,13 +131,22 @@ export class SessionManager {
|
||||
pendingMessages: [],
|
||||
abortController: new AbortController(),
|
||||
generatorPromise: null,
|
||||
lastPromptNumber: promptNumber || this.dbManager.getSessionStore().getPromptCounter(sessionDbId),
|
||||
lastPromptNumber: promptNumber || this.dbManager.getSessionStore().getPromptNumberFromUserPrompts(dbSession.claude_session_id),
|
||||
startTime: Date.now(),
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0,
|
||||
pendingProcessingIds: new Set()
|
||||
pendingProcessingIds: new Set(),
|
||||
earliestPendingTimestamp: null,
|
||||
conversationHistory: [], // Initialize empty - will be populated by agents
|
||||
currentProvider: null // Will be set when generator starts
|
||||
};
|
||||
|
||||
logger.info('SESSION', 'Creating new session object', {
|
||||
sessionDbId,
|
||||
claudeSessionId: dbSession.claude_session_id,
|
||||
lastPromptNumber: promptNumber || this.dbManager.getSessionStore().getPromptNumberFromUserPrompts(dbSession.claude_session_id)
|
||||
});
|
||||
|
||||
this.sessions.set(sessionDbId, session);
|
||||
|
||||
// Create event emitter for queue notifications
|
||||
@@ -445,6 +472,14 @@ export class SessionManager {
|
||||
// Track this message ID for completion marking
|
||||
session.pendingProcessingIds.add(persistentMessage.id);
|
||||
|
||||
// Track earliest timestamp for accurate observation timestamps
|
||||
// This ensures backlog messages get their original timestamps, not current time
|
||||
if (session.earliestPendingTimestamp === null) {
|
||||
session.earliestPendingTimestamp = persistentMessage.created_at_epoch;
|
||||
} else {
|
||||
session.earliestPendingTimestamp = Math.min(session.earliestPendingTimestamp, persistentMessage.created_at_epoch);
|
||||
}
|
||||
|
||||
// Convert to PendingMessageWithId and yield
|
||||
// Include original timestamp for accurate observation timestamps (survives stuck processing)
|
||||
const message: PendingMessageWithId = {
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
|
||||
import { ObservationSearchResult, SessionSummarySearchResult, UserPromptSearchResult } from '../sqlite/types.js';
|
||||
import { ModeManager } from '../domain/ModeManager.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
|
||||
/**
|
||||
* Timeline item for unified chronological display
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
|
||||
import { SSEBroadcaster } from '../SSEBroadcaster.js';
|
||||
import type { WorkerService } from '../../worker-service.js';
|
||||
import { logger } from '../../../utils/logger.js';
|
||||
|
||||
export class SessionEventBroadcaster {
|
||||
constructor(
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
import express, { Request, Response } from 'express';
|
||||
import path from 'path';
|
||||
import { readFileSync, statSync, existsSync } from 'fs';
|
||||
import { logger } from '../../../../utils/logger.js';
|
||||
import { homedir } from 'os';
|
||||
import { getPackageRoot } from '../../../../shared/paths.js';
|
||||
import { getWorkerPort } from '../../../../shared/worker-utils.js';
|
||||
@@ -51,6 +52,10 @@ export class DataRoutes extends BaseRouteHandler {
|
||||
app.get('/api/processing-status', this.handleGetProcessingStatus.bind(this));
|
||||
app.post('/api/processing', this.handleSetProcessing.bind(this));
|
||||
|
||||
// Pending queue management endpoints
|
||||
app.get('/api/pending-queue', this.handleGetPendingQueue.bind(this));
|
||||
app.post('/api/pending-queue/process', this.handleProcessPendingQueue.bind(this));
|
||||
|
||||
// Import endpoint
|
||||
app.post('/api/import', this.handleImport.bind(this));
|
||||
}
|
||||
@@ -364,4 +369,58 @@ export class DataRoutes extends BaseRouteHandler {
|
||||
stats
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Get pending queue contents
|
||||
* GET /api/pending-queue
|
||||
* Returns all pending, processing, and failed messages with optional recently processed
|
||||
*/
|
||||
private handleGetPendingQueue = this.wrapHandler((req: Request, res: Response): void => {
|
||||
const { PendingMessageStore } = require('../../../sqlite/PendingMessageStore.js');
|
||||
const pendingStore = new PendingMessageStore(this.dbManager.getSessionStore().db, 3);
|
||||
|
||||
// Get queue contents (pending, processing, failed)
|
||||
const queueMessages = pendingStore.getQueueMessages();
|
||||
|
||||
// Get recently processed (last 30 min, up to 20)
|
||||
const recentlyProcessed = pendingStore.getRecentlyProcessed(20, 30);
|
||||
|
||||
// Get stuck message count (processing > 5 min)
|
||||
const stuckCount = pendingStore.getStuckCount(5 * 60 * 1000);
|
||||
|
||||
// Get sessions with pending work
|
||||
const sessionsWithPending = pendingStore.getSessionsWithPendingMessages();
|
||||
|
||||
res.json({
|
||||
queue: {
|
||||
messages: queueMessages,
|
||||
totalPending: queueMessages.filter((m: { status: string }) => m.status === 'pending').length,
|
||||
totalProcessing: queueMessages.filter((m: { status: string }) => m.status === 'processing').length,
|
||||
totalFailed: queueMessages.filter((m: { status: string }) => m.status === 'failed').length,
|
||||
stuckCount
|
||||
},
|
||||
recentlyProcessed,
|
||||
sessionsWithPendingWork: sessionsWithPending
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Process pending queue
|
||||
* POST /api/pending-queue/process
|
||||
* Body: { sessionLimit?: number } - defaults to 10
|
||||
* Starts SDK agents for sessions with pending messages
|
||||
*/
|
||||
private handleProcessPendingQueue = this.wrapHandler(async (req: Request, res: Response): Promise<void> => {
|
||||
const sessionLimit = Math.min(
|
||||
Math.max(parseInt(req.body.sessionLimit, 10) || 10, 1),
|
||||
100 // Max 100 sessions at once
|
||||
);
|
||||
|
||||
const result = await this.workerService.processPendingQueues(sessionLimit);
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
...result
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
import express, { Request, Response } from 'express';
|
||||
import { SearchManager } from '../../SearchManager.js';
|
||||
import { BaseRouteHandler } from '../BaseRouteHandler.js';
|
||||
import { logger } from '../../../../utils/logger.js';
|
||||
|
||||
export class SearchRoutes extends BaseRouteHandler {
|
||||
constructor(
|
||||
|
||||
@@ -12,6 +12,8 @@ import { stripMemoryTagsFromJson, stripMemoryTagsFromPrompt } from '../../../../
|
||||
import { SessionManager } from '../../SessionManager.js';
|
||||
import { DatabaseManager } from '../../DatabaseManager.js';
|
||||
import { SDKAgent } from '../../SDKAgent.js';
|
||||
import { GeminiAgent, isGeminiSelected, isGeminiAvailable } from '../../GeminiAgent.js';
|
||||
import { OpenRouterAgent, isOpenRouterSelected, isOpenRouterAvailable } from '../../OpenRouterAgent.js';
|
||||
import type { WorkerService } from '../../../worker-service.js';
|
||||
import { BaseRouteHandler } from '../BaseRouteHandler.js';
|
||||
import { SessionEventBroadcaster } from '../../events/SessionEventBroadcaster.js';
|
||||
@@ -27,36 +29,125 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
private sessionManager: SessionManager,
|
||||
private dbManager: DatabaseManager,
|
||||
private sdkAgent: SDKAgent,
|
||||
private geminiAgent: GeminiAgent,
|
||||
private openRouterAgent: OpenRouterAgent,
|
||||
private eventBroadcaster: SessionEventBroadcaster,
|
||||
private workerService: WorkerService
|
||||
) {
|
||||
super();
|
||||
this.completionHandler = new SessionCompletionHandler(
|
||||
sessionManager,
|
||||
dbManager,
|
||||
eventBroadcaster
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensures SDK agent generator is running for a session
|
||||
* Get the appropriate agent based on settings
|
||||
* Throws error if provider is selected but not configured (no silent fallback)
|
||||
*
|
||||
* Note: Session linking via claudeSessionId allows provider switching mid-session.
|
||||
* The conversationHistory on ActiveSession maintains context across providers.
|
||||
*/
|
||||
private getActiveAgent(): SDKAgent | GeminiAgent | OpenRouterAgent {
|
||||
if (isOpenRouterSelected()) {
|
||||
if (isOpenRouterAvailable()) {
|
||||
logger.debug('SESSION', 'Using OpenRouter agent');
|
||||
return this.openRouterAgent;
|
||||
} else {
|
||||
throw new Error('OpenRouter provider selected but no API key configured. Set CLAUDE_MEM_OPENROUTER_API_KEY in settings or OPENROUTER_API_KEY environment variable.');
|
||||
}
|
||||
}
|
||||
if (isGeminiSelected()) {
|
||||
if (isGeminiAvailable()) {
|
||||
logger.debug('SESSION', 'Using Gemini agent');
|
||||
return this.geminiAgent;
|
||||
} else {
|
||||
throw new Error('Gemini provider selected but no API key configured. Set CLAUDE_MEM_GEMINI_API_KEY in settings or GEMINI_API_KEY environment variable.');
|
||||
}
|
||||
}
|
||||
return this.sdkAgent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the currently selected provider name
|
||||
*/
|
||||
private getSelectedProvider(): 'claude' | 'gemini' | 'openrouter' {
|
||||
if (isOpenRouterSelected() && isOpenRouterAvailable()) {
|
||||
return 'openrouter';
|
||||
}
|
||||
return (isGeminiSelected() && isGeminiAvailable()) ? 'gemini' : 'claude';
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensures agent generator is running for a session
|
||||
* Auto-starts if not already running to process pending queue
|
||||
* Uses either Claude SDK or Gemini based on settings
|
||||
*
|
||||
* Provider switching: If provider setting changed while generator is running,
|
||||
* we let the current generator finish naturally (max 5s linger timeout).
|
||||
* The next generator will use the new provider with shared conversationHistory.
|
||||
*/
|
||||
private ensureGeneratorRunning(sessionDbId: number, source: string): void {
|
||||
const session = this.sessionManager.getSession(sessionDbId);
|
||||
if (session && !session.generatorPromise) {
|
||||
logger.info('SESSION', `Generator auto-starting (${source})`, {
|
||||
sessionId: sessionDbId,
|
||||
queueDepth: session.pendingMessages.length
|
||||
});
|
||||
if (!session) return;
|
||||
|
||||
session.generatorPromise = this.sdkAgent.startSession(session, this.workerService)
|
||||
.finally(() => {
|
||||
logger.info('SESSION', `Generator finished`, { sessionId: sessionDbId });
|
||||
session.generatorPromise = null;
|
||||
this.workerService.broadcastProcessingStatus();
|
||||
});
|
||||
const selectedProvider = this.getSelectedProvider();
|
||||
|
||||
// Start generator if not running
|
||||
if (!session.generatorPromise) {
|
||||
this.startGeneratorWithProvider(session, selectedProvider, source);
|
||||
return;
|
||||
}
|
||||
|
||||
// Generator is running - check if provider changed
|
||||
if (session.currentProvider && session.currentProvider !== selectedProvider) {
|
||||
logger.info('SESSION', `Provider changed, will switch after current generator finishes`, {
|
||||
sessionId: sessionDbId,
|
||||
currentProvider: session.currentProvider,
|
||||
selectedProvider,
|
||||
historyLength: session.conversationHistory.length
|
||||
});
|
||||
// Let current generator finish naturally, next one will use new provider
|
||||
// The shared conversationHistory ensures context is preserved
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Start a generator with the specified provider
|
||||
*/
|
||||
private startGeneratorWithProvider(
|
||||
session: ReturnType<typeof this.sessionManager.getSession>,
|
||||
provider: 'claude' | 'gemini' | 'openrouter',
|
||||
source: string
|
||||
): void {
|
||||
if (!session) return;
|
||||
|
||||
const agent = provider === 'openrouter' ? this.openRouterAgent : (provider === 'gemini' ? this.geminiAgent : this.sdkAgent);
|
||||
const agentName = provider === 'openrouter' ? 'OpenRouter' : (provider === 'gemini' ? 'Gemini' : 'Claude SDK');
|
||||
|
||||
logger.info('SESSION', `Generator auto-starting (${source}) using ${agentName}`, {
|
||||
sessionId: session.sessionDbId,
|
||||
queueDepth: session.pendingMessages.length,
|
||||
historyLength: session.conversationHistory.length
|
||||
});
|
||||
|
||||
// Track which provider is running
|
||||
session.currentProvider = provider;
|
||||
|
||||
session.generatorPromise = agent.startSession(session, this.workerService)
|
||||
.catch(error => {
|
||||
logger.error('SESSION', `Generator failed`, {
|
||||
sessionId: session.sessionDbId,
|
||||
provider: provider,
|
||||
error: error.message
|
||||
}, error);
|
||||
})
|
||||
.finally(() => {
|
||||
logger.info('SESSION', `Generator finished`, { sessionId: session.sessionDbId });
|
||||
session.generatorPromise = null;
|
||||
session.currentProvider = null;
|
||||
this.workerService.broadcastProcessingStatus();
|
||||
});
|
||||
}
|
||||
|
||||
setupRoutes(app: express.Application): void {
|
||||
@@ -72,7 +163,6 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
app.post('/api/sessions/init', this.handleSessionInitByClaudeId.bind(this));
|
||||
app.post('/api/sessions/observations', this.handleObservationsByClaudeId.bind(this));
|
||||
app.post('/api/sessions/summarize', this.handleSummarizeByClaudeId.bind(this));
|
||||
app.post('/api/sessions/complete', this.handleSessionCompleteByClaudeId.bind(this));
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -83,6 +173,12 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
if (sessionDbId === null) return;
|
||||
|
||||
const { userPrompt, promptNumber } = req.body;
|
||||
logger.info('HTTP', 'SessionRoutes: handleSessionInit called', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
has_userPrompt: !!userPrompt
|
||||
});
|
||||
|
||||
const session = this.sessionManager.initializeSession(sessionDbId, userPrompt, promptNumber);
|
||||
|
||||
// Get the latest user_prompt for this session to sync to Chroma
|
||||
@@ -119,24 +215,16 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
duration: `${chromaDuration}ms`,
|
||||
prompt: truncatedPrompt
|
||||
});
|
||||
}).catch((error) => {
|
||||
logger.warn('CHROMA', 'User prompt sync failed, continuing without vector search', {
|
||||
promptId: latestPrompt.id,
|
||||
prompt: promptText.length > 60 ? promptText.substring(0, 60) + '...' : promptText
|
||||
}, error);
|
||||
});
|
||||
}
|
||||
|
||||
// Start SDK agent in background (pass worker ref for spinner control)
|
||||
logger.info('SESSION', 'Generator starting', {
|
||||
sessionId: sessionDbId,
|
||||
project: session.project,
|
||||
promptNum: session.lastPromptNumber
|
||||
});
|
||||
|
||||
session.generatorPromise = this.sdkAgent.startSession(session, this.workerService)
|
||||
.finally(() => {
|
||||
// Clear generator reference when completed
|
||||
logger.info('SESSION', `Generator finished`, { sessionId: sessionDbId });
|
||||
session.generatorPromise = null;
|
||||
// Broadcast status change (generator finished, may stop spinner)
|
||||
this.workerService.broadcastProcessingStatus();
|
||||
});
|
||||
// Start agent in background using the helper method
|
||||
this.startGeneratorWithProvider(session, this.getSelectedProvider(), 'init');
|
||||
|
||||
// Broadcast session started event
|
||||
this.eventBroadcaster.broadcastSessionStarted(sessionDbId, session.project);
|
||||
@@ -281,7 +369,7 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
|
||||
// Get or create session
|
||||
const sessionDbId = store.createSDKSession(claudeSessionId, '', '');
|
||||
const promptNumber = store.getPromptCounter(sessionDbId);
|
||||
const promptNumber = store.getPromptNumberFromUserPrompts(claudeSessionId);
|
||||
|
||||
// Privacy check: skip if user prompt was entirely private
|
||||
const userPrompt = PrivacyCheckValidator.checkUserPromptPrivacy(
|
||||
@@ -348,7 +436,7 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
|
||||
// Get or create session
|
||||
const sessionDbId = store.createSDKSession(claudeSessionId, '', '');
|
||||
const promptNumber = store.getPromptCounter(sessionDbId);
|
||||
const promptNumber = store.getPromptNumberFromUserPrompts(claudeSessionId);
|
||||
|
||||
// Privacy check: skip if user prompt was entirely private
|
||||
const userPrompt = PrivacyCheckValidator.checkUserPromptPrivacy(
|
||||
@@ -385,31 +473,6 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
res.json({ status: 'queued' });
|
||||
});
|
||||
|
||||
/**
|
||||
* Complete session by claudeSessionId (cleanup-hook uses this)
|
||||
* POST /api/sessions/complete
|
||||
* Body: { claudeSessionId }
|
||||
*
|
||||
* Marks session complete, stops SDK agent, broadcasts status
|
||||
*/
|
||||
private handleSessionCompleteByClaudeId = this.wrapHandler(async (req: Request, res: Response): Promise<void> => {
|
||||
const { claudeSessionId } = req.body;
|
||||
|
||||
if (!claudeSessionId) {
|
||||
return this.badRequest(res, 'Missing claudeSessionId');
|
||||
}
|
||||
|
||||
const found = await this.completionHandler.completeByClaudeId(claudeSessionId);
|
||||
|
||||
if (!found) {
|
||||
// No active session - nothing to clean up (may have already been completed)
|
||||
res.json({ success: true, message: 'No active session found' });
|
||||
return;
|
||||
}
|
||||
|
||||
res.json({ success: true });
|
||||
});
|
||||
|
||||
/**
|
||||
* Initialize session by claudeSessionId (new-hook uses this)
|
||||
* POST /api/sessions/init
|
||||
@@ -425,6 +488,12 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
private handleSessionInitByClaudeId = this.wrapHandler((req: Request, res: Response): void => {
|
||||
const { claudeSessionId, project, prompt } = req.body;
|
||||
|
||||
logger.info('HTTP', 'SessionRoutes: handleSessionInitByClaudeId called', {
|
||||
claudeSessionId,
|
||||
project,
|
||||
prompt_length: prompt?.length
|
||||
});
|
||||
|
||||
// Validate required parameters
|
||||
if (!this.validateRequired(req, res, ['claudeSessionId', 'project', 'prompt'])) {
|
||||
return;
|
||||
@@ -435,8 +504,20 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
// Step 1: Create/get SDK session (idempotent INSERT OR IGNORE)
|
||||
const sessionDbId = store.createSDKSession(claudeSessionId, project, prompt);
|
||||
|
||||
// Step 2: Increment prompt counter
|
||||
const promptNumber = store.incrementPromptCounter(sessionDbId);
|
||||
logger.info('HTTP', 'SessionRoutes: createSDKSession returned', {
|
||||
sessionDbId,
|
||||
claudeSessionId
|
||||
});
|
||||
|
||||
// Step 2: Get next prompt number from user_prompts count
|
||||
const currentCount = store.getPromptNumberFromUserPrompts(claudeSessionId);
|
||||
const promptNumber = currentCount + 1;
|
||||
|
||||
logger.info('HTTP', 'SessionRoutes: Calculated promptNumber', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
currentCount
|
||||
});
|
||||
|
||||
// Step 3: Strip privacy tags from prompt
|
||||
const cleanedPrompt = stripMemoryTagsFromPrompt(prompt);
|
||||
|
||||
@@ -71,7 +71,16 @@ export class SettingsRoutes extends BaseRouteHandler {
|
||||
|
||||
if (existsSync(settingsPath)) {
|
||||
const settingsData = readFileSync(settingsPath, 'utf-8');
|
||||
settings = JSON.parse(settingsData);
|
||||
try {
|
||||
settings = JSON.parse(settingsData);
|
||||
} catch (parseError) {
|
||||
logger.error('SETTINGS', 'Failed to parse settings file', { settingsPath }, parseError as Error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: 'Settings file is corrupted. Delete ~/.claude-mem/settings.json to reset.'
|
||||
});
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// Update all settings from request body
|
||||
@@ -80,6 +89,18 @@ export class SettingsRoutes extends BaseRouteHandler {
|
||||
'CLAUDE_MEM_CONTEXT_OBSERVATIONS',
|
||||
'CLAUDE_MEM_WORKER_PORT',
|
||||
'CLAUDE_MEM_WORKER_HOST',
|
||||
// AI Provider Configuration
|
||||
'CLAUDE_MEM_PROVIDER',
|
||||
'CLAUDE_MEM_GEMINI_API_KEY',
|
||||
'CLAUDE_MEM_GEMINI_MODEL',
|
||||
'CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED',
|
||||
// OpenRouter Configuration
|
||||
'CLAUDE_MEM_OPENROUTER_API_KEY',
|
||||
'CLAUDE_MEM_OPENROUTER_MODEL',
|
||||
'CLAUDE_MEM_OPENROUTER_SITE_URL',
|
||||
'CLAUDE_MEM_OPENROUTER_APP_NAME',
|
||||
'CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES',
|
||||
'CLAUDE_MEM_OPENROUTER_MAX_TOKENS',
|
||||
// System Configuration
|
||||
'CLAUDE_MEM_DATA_DIR',
|
||||
'CLAUDE_MEM_LOG_LEVEL',
|
||||
@@ -210,6 +231,22 @@ export class SettingsRoutes extends BaseRouteHandler {
|
||||
* Validate all settings from request body (single source of truth)
|
||||
*/
|
||||
private validateSettings(settings: any): { valid: boolean; error?: string } {
|
||||
// Validate CLAUDE_MEM_PROVIDER
|
||||
if (settings.CLAUDE_MEM_PROVIDER) {
|
||||
const validProviders = ['claude', 'gemini', 'openrouter'];
|
||||
if (!validProviders.includes(settings.CLAUDE_MEM_PROVIDER)) {
|
||||
return { valid: false, error: 'CLAUDE_MEM_PROVIDER must be "claude", "gemini", or "openrouter"' };
|
||||
}
|
||||
}
|
||||
|
||||
// Validate CLAUDE_MEM_GEMINI_MODEL
|
||||
if (settings.CLAUDE_MEM_GEMINI_MODEL) {
|
||||
const validGeminiModels = ['gemini-2.5-flash-lite', 'gemini-2.5-flash', 'gemini-3-flash'];
|
||||
if (!validGeminiModels.includes(settings.CLAUDE_MEM_GEMINI_MODEL)) {
|
||||
return { valid: false, error: 'CLAUDE_MEM_GEMINI_MODEL must be one of: gemini-2.5-flash-lite, gemini-2.5-flash, gemini-3-flash' };
|
||||
}
|
||||
}
|
||||
|
||||
// Validate CLAUDE_MEM_CONTEXT_OBSERVATIONS
|
||||
if (settings.CLAUDE_MEM_CONTEXT_OBSERVATIONS) {
|
||||
const obsCount = parseInt(settings.CLAUDE_MEM_CONTEXT_OBSERVATIONS, 10);
|
||||
@@ -291,6 +328,31 @@ export class SettingsRoutes extends BaseRouteHandler {
|
||||
}
|
||||
}
|
||||
|
||||
// Validate CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES
|
||||
if (settings.CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES) {
|
||||
const count = parseInt(settings.CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES, 10);
|
||||
if (isNaN(count) || count < 1 || count > 100) {
|
||||
return { valid: false, error: 'CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES must be between 1 and 100' };
|
||||
}
|
||||
}
|
||||
|
||||
// Validate CLAUDE_MEM_OPENROUTER_MAX_TOKENS
|
||||
if (settings.CLAUDE_MEM_OPENROUTER_MAX_TOKENS) {
|
||||
const tokens = parseInt(settings.CLAUDE_MEM_OPENROUTER_MAX_TOKENS, 10);
|
||||
if (isNaN(tokens) || tokens < 1000 || tokens > 1000000) {
|
||||
return { valid: false, error: 'CLAUDE_MEM_OPENROUTER_MAX_TOKENS must be between 1000 and 1000000' };
|
||||
}
|
||||
}
|
||||
|
||||
// Validate CLAUDE_MEM_OPENROUTER_SITE_URL if provided
|
||||
if (settings.CLAUDE_MEM_OPENROUTER_SITE_URL) {
|
||||
try {
|
||||
new URL(settings.CLAUDE_MEM_OPENROUTER_SITE_URL);
|
||||
} catch {
|
||||
return { valid: false, error: 'CLAUDE_MEM_OPENROUTER_SITE_URL must be a valid URL' };
|
||||
}
|
||||
}
|
||||
|
||||
// Skip observation types validation - any type string is valid since modes define their own types
|
||||
// The database accepts any TEXT value, and mode-specific validation happens at parse time
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
import express, { Request, Response } from 'express';
|
||||
import path from 'path';
|
||||
import { readFileSync, existsSync } from 'fs';
|
||||
import { logger } from '../../../../utils/logger.js';
|
||||
import { getPackageRoot } from '../../../../shared/paths.js';
|
||||
import { SSEBroadcaster } from '../../SSEBroadcaster.js';
|
||||
import { DatabaseManager } from '../../DatabaseManager.js';
|
||||
|
||||
@@ -1,23 +1,21 @@
|
||||
/**
|
||||
* Session Completion Handler
|
||||
*
|
||||
* Consolidates session completion logic to eliminate duplication across
|
||||
* three different completion endpoints (DELETE, POST by DB ID, POST by Claude ID).
|
||||
* Consolidates session completion logic for manual session deletion/completion.
|
||||
* Used by DELETE /api/sessions/:id and POST /api/sessions/:id/complete endpoints.
|
||||
*
|
||||
* All completion flows follow the same pattern:
|
||||
* 1. Delete session from SessionManager (aborts SDK agent)
|
||||
* 2. Mark session complete in database
|
||||
* 3. Broadcast session completed event
|
||||
* Completion flow:
|
||||
* 1. Delete session from SessionManager (aborts SDK agent, cleans up in-memory state)
|
||||
* 2. Broadcast session completed event (updates UI spinner)
|
||||
*/
|
||||
|
||||
import { SessionManager } from '../SessionManager.js';
|
||||
import { DatabaseManager } from '../DatabaseManager.js';
|
||||
import { SessionEventBroadcaster } from '../events/SessionEventBroadcaster.js';
|
||||
import { logger } from '../../../utils/logger.js';
|
||||
|
||||
export class SessionCompletionHandler {
|
||||
constructor(
|
||||
private sessionManager: SessionManager,
|
||||
private dbManager: DatabaseManager,
|
||||
private eventBroadcaster: SessionEventBroadcaster
|
||||
) {}
|
||||
|
||||
@@ -29,34 +27,7 @@ export class SessionCompletionHandler {
|
||||
// Delete from session manager (aborts SDK agent)
|
||||
await this.sessionManager.deleteSession(sessionDbId);
|
||||
|
||||
// Mark session complete in database
|
||||
this.dbManager.markSessionComplete(sessionDbId);
|
||||
|
||||
// Broadcast session completed event
|
||||
this.eventBroadcaster.broadcastSessionCompleted(sessionDbId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Complete session by Claude session ID
|
||||
* Used by POST /api/sessions/complete (cleanup-hook endpoint)
|
||||
*
|
||||
* @returns true if session was found and completed, false if no active session found
|
||||
*/
|
||||
async completeByClaudeId(claudeSessionId: string): Promise<boolean> {
|
||||
const store = this.dbManager.getSessionStore();
|
||||
|
||||
// Find session by claudeSessionId
|
||||
const session = store.findActiveSDKSession(claudeSessionId);
|
||||
if (!session) {
|
||||
// No active session - nothing to clean up (may have already been completed)
|
||||
return false;
|
||||
}
|
||||
|
||||
const sessionDbId = session.id;
|
||||
|
||||
// Complete using standard flow
|
||||
await this.completeByDbId(sessionDbId);
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,6 +17,17 @@ export interface SettingsDefaults {
|
||||
CLAUDE_MEM_WORKER_PORT: string;
|
||||
CLAUDE_MEM_WORKER_HOST: string;
|
||||
CLAUDE_MEM_SKIP_TOOLS: string;
|
||||
// AI Provider Configuration
|
||||
CLAUDE_MEM_PROVIDER: string; // 'claude' | 'gemini' | 'openrouter'
|
||||
CLAUDE_MEM_GEMINI_API_KEY: string;
|
||||
CLAUDE_MEM_GEMINI_MODEL: string; // 'gemini-2.5-flash-lite' | 'gemini-2.5-flash' | 'gemini-3-flash'
|
||||
CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED: string; // 'true' | 'false' - enable rate limiting for free tier
|
||||
CLAUDE_MEM_OPENROUTER_API_KEY: string;
|
||||
CLAUDE_MEM_OPENROUTER_MODEL: string;
|
||||
CLAUDE_MEM_OPENROUTER_SITE_URL: string;
|
||||
CLAUDE_MEM_OPENROUTER_APP_NAME: string;
|
||||
CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES: string;
|
||||
CLAUDE_MEM_OPENROUTER_MAX_TOKENS: string;
|
||||
// System Configuration
|
||||
CLAUDE_MEM_DATA_DIR: string;
|
||||
CLAUDE_MEM_LOG_LEVEL: string;
|
||||
@@ -50,6 +61,17 @@ export class SettingsDefaultsManager {
|
||||
CLAUDE_MEM_WORKER_PORT: '37777',
|
||||
CLAUDE_MEM_WORKER_HOST: '127.0.0.1',
|
||||
CLAUDE_MEM_SKIP_TOOLS: 'ListMcpResourcesTool,SlashCommand,Skill,TodoWrite,AskUserQuestion',
|
||||
// AI Provider Configuration
|
||||
CLAUDE_MEM_PROVIDER: 'claude', // Default to Claude
|
||||
CLAUDE_MEM_GEMINI_API_KEY: '', // Empty by default, can be set via UI or env
|
||||
CLAUDE_MEM_GEMINI_MODEL: 'gemini-2.5-flash-lite', // Default Gemini model (highest free tier RPM)
|
||||
CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED: 'true', // Rate limiting ON by default for free tier users
|
||||
CLAUDE_MEM_OPENROUTER_API_KEY: '', // Empty by default, can be set via UI or env
|
||||
CLAUDE_MEM_OPENROUTER_MODEL: 'xiaomi/mimo-v2-flash:free', // Default OpenRouter model (free tier)
|
||||
CLAUDE_MEM_OPENROUTER_SITE_URL: '', // Optional: for OpenRouter analytics
|
||||
CLAUDE_MEM_OPENROUTER_APP_NAME: 'claude-mem', // App name for OpenRouter analytics
|
||||
CLAUDE_MEM_OPENROUTER_MAX_CONTEXT_MESSAGES: '20', // Max messages in context window
|
||||
CLAUDE_MEM_OPENROUTER_MAX_TOKENS: '100000', // Max estimated tokens (~100k safety limit)
|
||||
// System Configuration
|
||||
CLAUDE_MEM_DATA_DIR: join(homedir(), '.claude-mem'),
|
||||
CLAUDE_MEM_LOG_LEVEL: 'INFO',
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
export const HOOK_TIMEOUTS = {
|
||||
DEFAULT: 5000, // Standard HTTP timeout (up from 2000ms)
|
||||
HEALTH_CHECK: 1000, // Worker health check (up from 500ms)
|
||||
DEFAULT: 300000, // Standard HTTP timeout (5 min for slow systems)
|
||||
HEALTH_CHECK: 30000, // Worker health check (30s for slow systems)
|
||||
WORKER_STARTUP_WAIT: 1000,
|
||||
WORKER_STARTUP_RETRIES: 15,
|
||||
WORKER_STARTUP_RETRIES: 300,
|
||||
PRE_RESTART_SETTLE_DELAY: 2000, // Give files time to sync before restart
|
||||
WINDOWS_MULTIPLIER: 1.5 // Platform-specific adjustment
|
||||
} as const;
|
||||
|
||||
+18
-99
@@ -1,10 +1,8 @@
|
||||
import path from "path";
|
||||
import { homedir } from "os";
|
||||
import { spawnSync } from "child_process";
|
||||
import { existsSync, writeFileSync, readFileSync, mkdirSync } from "fs";
|
||||
import { readFileSync } from "fs";
|
||||
import { logger } from "../utils/logger.js";
|
||||
import { HOOK_TIMEOUTS, getTimeout } from "./hook-constants.js";
|
||||
import { ProcessManager } from "../services/process/ProcessManager.js";
|
||||
import { SettingsDefaultsManager } from "./SettingsDefaultsManager.js";
|
||||
import { getWorkerRestartInstructions } from "../utils/error-messages.js";
|
||||
|
||||
@@ -96,123 +94,44 @@ async function getWorkerVersion(): Promise<string> {
|
||||
|
||||
/**
|
||||
* Check if worker version matches plugin version
|
||||
* If mismatch detected, restart the worker automatically
|
||||
* Logs a warning if mismatch is detected
|
||||
*/
|
||||
async function ensureWorkerVersionMatches(): Promise<void> {
|
||||
async function checkWorkerVersion(): Promise<void> {
|
||||
const pluginVersion = getPluginVersion();
|
||||
const workerVersion = await getWorkerVersion();
|
||||
|
||||
if (pluginVersion !== workerVersion) {
|
||||
logger.info('SYSTEM', 'Worker version mismatch detected - restarting worker', {
|
||||
logger.warn('SYSTEM', 'Worker version mismatch', {
|
||||
pluginVersion,
|
||||
workerVersion
|
||||
workerVersion,
|
||||
hint: 'Restart worker with: claude-mem worker restart'
|
||||
});
|
||||
|
||||
// Give files time to sync before restart
|
||||
await new Promise(resolve => setTimeout(resolve, getTimeout(HOOK_TIMEOUTS.PRE_RESTART_SETTLE_DELAY)));
|
||||
|
||||
// Restart the worker
|
||||
await ProcessManager.restart(getWorkerPort());
|
||||
|
||||
// Give it a moment to start
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
|
||||
// Verify it's healthy
|
||||
if (!await isWorkerHealthy()) {
|
||||
throw new Error(`Worker failed to restart after version mismatch. Expected ${pluginVersion}, was running ${workerVersion}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the worker service using ProcessManager
|
||||
* Handles both Unix (Bun) and Windows (compiled exe) platforms
|
||||
*/
|
||||
async function startWorker(): Promise<boolean> {
|
||||
// Clean up legacy PM2 (one-time migration)
|
||||
const dataDir = SettingsDefaultsManager.get('CLAUDE_MEM_DATA_DIR');
|
||||
const pm2MigratedMarker = path.join(dataDir, '.pm2-migrated');
|
||||
|
||||
// Ensure data directory exists (may not exist on fresh install)
|
||||
mkdirSync(dataDir, { recursive: true });
|
||||
|
||||
if (!existsSync(pm2MigratedMarker)) {
|
||||
spawnSync('pm2', ['delete', 'claude-mem-worker'], { stdio: 'ignore' });
|
||||
// Mark migration as complete
|
||||
writeFileSync(pm2MigratedMarker, new Date().toISOString(), 'utf-8');
|
||||
logger.debug('SYSTEM', 'PM2 cleanup completed and marked');
|
||||
}
|
||||
|
||||
const port = getWorkerPort();
|
||||
const result = await ProcessManager.start(port);
|
||||
|
||||
if (!result.success) {
|
||||
logger.error('SYSTEM', 'Failed to start worker', {
|
||||
platform: process.platform,
|
||||
port,
|
||||
error: result.error,
|
||||
marketplaceRoot: MARKETPLACE_ROOT
|
||||
});
|
||||
}
|
||||
|
||||
return result.success;
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure worker service is running
|
||||
* Checks health and auto-starts if not running
|
||||
* Also ensures worker version matches plugin version
|
||||
* Polls until worker is ready (assumes worker-cli.js start was called by hooks.json)
|
||||
*/
|
||||
export async function ensureWorkerRunning(): Promise<void> {
|
||||
// Check if already healthy (will throw on fetch errors)
|
||||
let healthy = false;
|
||||
try {
|
||||
healthy = await isWorkerHealthy();
|
||||
} catch (error) {
|
||||
// Worker not running or unreachable - continue to start it
|
||||
healthy = false;
|
||||
}
|
||||
const maxRetries = 25; // 5 seconds total
|
||||
const pollInterval = 200;
|
||||
|
||||
if (healthy) {
|
||||
// Worker is healthy, but check if version matches
|
||||
await ensureWorkerVersionMatches();
|
||||
return;
|
||||
}
|
||||
|
||||
// Try to start the worker
|
||||
const started = await startWorker();
|
||||
|
||||
if (!started) {
|
||||
const port = getWorkerPort();
|
||||
throw new Error(
|
||||
getWorkerRestartInstructions({
|
||||
port,
|
||||
customPrefix: `Worker service failed to start on port ${port}.`
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
// Wait for worker to become responsive after starting
|
||||
// Try up to 5 times with 500ms delays (2.5 seconds total)
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await new Promise(resolve => setTimeout(resolve, 500));
|
||||
for (let i = 0; i < maxRetries; i++) {
|
||||
try {
|
||||
if (await isWorkerHealthy()) {
|
||||
await ensureWorkerVersionMatches();
|
||||
await checkWorkerVersion(); // logs warning on mismatch, doesn't restart
|
||||
return;
|
||||
}
|
||||
} catch (error) {
|
||||
// Continue trying
|
||||
} catch {
|
||||
// Continue polling
|
||||
}
|
||||
await new Promise(r => setTimeout(r, pollInterval));
|
||||
}
|
||||
|
||||
// Worker started but isn't responding
|
||||
const port = getWorkerPort();
|
||||
logger.error('SYSTEM', 'Worker started but not responding to health checks');
|
||||
throw new Error(
|
||||
getWorkerRestartInstructions({
|
||||
port,
|
||||
customPrefix: `Worker service started but is not responding on port ${port}.`
|
||||
})
|
||||
);
|
||||
throw new Error(getWorkerRestartInstructions({
|
||||
port: getWorkerPort(),
|
||||
customPrefix: 'Worker did not become ready within 5 seconds.'
|
||||
}));
|
||||
}
|
||||
|
||||
@@ -1817,6 +1817,49 @@
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
.modal-footer {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
align-items: center;
|
||||
gap: 16px;
|
||||
padding: 16px 24px;
|
||||
border-top: 1px solid var(--modal-border);
|
||||
background: var(--modal-header-bg);
|
||||
}
|
||||
|
||||
.modal-footer .save-status {
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.modal-footer .save-status .success {
|
||||
color: var(--success-color, #22c55e);
|
||||
}
|
||||
|
||||
.modal-footer .save-status .error {
|
||||
color: var(--error-color, #ef4444);
|
||||
}
|
||||
|
||||
.modal-footer .save-btn {
|
||||
padding: 8px 24px;
|
||||
background: var(--accent-color, #3b82f6);
|
||||
color: white;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
font-size: 14px;
|
||||
font-weight: 500;
|
||||
cursor: pointer;
|
||||
transition: background 0.15s ease;
|
||||
}
|
||||
|
||||
.modal-footer .save-btn:hover:not(:disabled) {
|
||||
background: var(--accent-hover, #2563eb);
|
||||
}
|
||||
|
||||
.modal-footer .save-btn:disabled {
|
||||
opacity: 0.6;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
/* Preview Column - Terminal Style */
|
||||
.preview-column {
|
||||
padding: 20px;
|
||||
|
||||
@@ -12,15 +12,6 @@ interface ContextSettingsModalProps {
|
||||
saveStatus: string;
|
||||
}
|
||||
|
||||
// Simple debounce helper
|
||||
function debounce<T extends (...args: any[]) => any>(fn: T, ms: number): T {
|
||||
let timeoutId: NodeJS.Timeout;
|
||||
return ((...args: any[]) => {
|
||||
clearTimeout(timeoutId);
|
||||
timeoutId = setTimeout(() => fn(...args), ms);
|
||||
}) as T;
|
||||
}
|
||||
|
||||
// Collapsible section component
|
||||
function CollapsibleSection({
|
||||
title,
|
||||
@@ -195,14 +186,6 @@ export function ContextSettingsModal({
|
||||
}: ContextSettingsModalProps) {
|
||||
const [formState, setFormState] = useState<Settings>(settings);
|
||||
|
||||
// Create debounced save function
|
||||
const debouncedSave = useCallback(
|
||||
debounce((newSettings: Settings) => {
|
||||
onSave(newSettings);
|
||||
}, 300),
|
||||
[onSave]
|
||||
);
|
||||
|
||||
// Update form state when settings prop changes
|
||||
useEffect(() => {
|
||||
setFormState(settings);
|
||||
@@ -214,8 +197,11 @@ export function ContextSettingsModal({
|
||||
const updateSetting = useCallback((key: keyof Settings, value: string) => {
|
||||
const newState = { ...formState, [key]: value };
|
||||
setFormState(newState);
|
||||
debouncedSave(newState);
|
||||
}, [formState, debouncedSave]);
|
||||
}, [formState]);
|
||||
|
||||
const handleSave = useCallback(() => {
|
||||
onSave(formState);
|
||||
}, [formState, onSave]);
|
||||
|
||||
const toggleBoolean = useCallback((key: keyof Settings) => {
|
||||
const currentValue = formState[key];
|
||||
@@ -423,24 +409,126 @@ export function ContextSettingsModal({
|
||||
{/* Section 4: Advanced */}
|
||||
<CollapsibleSection
|
||||
title="Advanced"
|
||||
description="Model selection and integrations"
|
||||
description="AI provider and model selection"
|
||||
defaultOpen={false}
|
||||
>
|
||||
<FormField
|
||||
label="Model"
|
||||
tooltip="AI model used for generating observations"
|
||||
label="AI Provider"
|
||||
tooltip="Choose between Claude (via Agent SDK) or Gemini (via REST API)"
|
||||
>
|
||||
<select
|
||||
value={formState.CLAUDE_MEM_MODEL || 'haiku'}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_MODEL', e.target.value)}
|
||||
value={formState.CLAUDE_MEM_PROVIDER || 'claude'}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_PROVIDER', e.target.value)}
|
||||
>
|
||||
{/* Shorthand names forward to latest model version */}
|
||||
<option value="haiku">haiku (fastest)</option>
|
||||
<option value="sonnet">sonnet (balanced)</option>
|
||||
<option value="opus">opus (highest quality)</option>
|
||||
<option value="claude">Claude (uses your Claude account)</option>
|
||||
<option value="gemini">Gemini (uses API key)</option>
|
||||
<option value="openrouter">OpenRouter (multi-model)</option>
|
||||
</select>
|
||||
</FormField>
|
||||
|
||||
{formState.CLAUDE_MEM_PROVIDER === 'claude' && (
|
||||
<FormField
|
||||
label="Claude Model"
|
||||
tooltip="Claude model used for generating observations"
|
||||
>
|
||||
<select
|
||||
value={formState.CLAUDE_MEM_MODEL || 'haiku'}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_MODEL', e.target.value)}
|
||||
>
|
||||
<option value="haiku">haiku (fastest)</option>
|
||||
<option value="sonnet">sonnet (balanced)</option>
|
||||
<option value="opus">opus (highest quality)</option>
|
||||
</select>
|
||||
</FormField>
|
||||
)}
|
||||
|
||||
{formState.CLAUDE_MEM_PROVIDER === 'gemini' && (
|
||||
<>
|
||||
<FormField
|
||||
label="Gemini API Key"
|
||||
tooltip="Your Google AI Studio API key (or set GEMINI_API_KEY env var)"
|
||||
>
|
||||
<input
|
||||
type="password"
|
||||
value={formState.CLAUDE_MEM_GEMINI_API_KEY || ''}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_GEMINI_API_KEY', e.target.value)}
|
||||
placeholder="Enter Gemini API key..."
|
||||
/>
|
||||
</FormField>
|
||||
<FormField
|
||||
label="Gemini Model"
|
||||
tooltip="Gemini model used for generating observations"
|
||||
>
|
||||
<select
|
||||
value={formState.CLAUDE_MEM_GEMINI_MODEL || 'gemini-2.5-flash-lite'}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_GEMINI_MODEL', e.target.value)}
|
||||
>
|
||||
<option value="gemini-2.5-flash-lite">gemini-2.5-flash-lite (10 RPM free)</option>
|
||||
<option value="gemini-2.5-flash">gemini-2.5-flash (5 RPM free)</option>
|
||||
<option value="gemini-3-flash">gemini-3-flash (5 RPM free)</option>
|
||||
</select>
|
||||
</FormField>
|
||||
<div className="toggle-group" style={{ marginTop: '8px' }}>
|
||||
<ToggleSwitch
|
||||
id="gemini-rate-limiting"
|
||||
label="Rate Limiting"
|
||||
description="Enable for free tier (10-30 RPM). Disable if you have billing set up (1000+ RPM)."
|
||||
checked={formState.CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED === 'true'}
|
||||
onChange={(checked) => updateSetting('CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED', checked ? 'true' : 'false')}
|
||||
/>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
{formState.CLAUDE_MEM_PROVIDER === 'openrouter' && (
|
||||
<>
|
||||
<FormField
|
||||
label="OpenRouter API Key"
|
||||
tooltip="Your OpenRouter API key from openrouter.ai (or set OPENROUTER_API_KEY env var)"
|
||||
>
|
||||
<input
|
||||
type="password"
|
||||
value={formState.CLAUDE_MEM_OPENROUTER_API_KEY || ''}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_OPENROUTER_API_KEY', e.target.value)}
|
||||
placeholder="Enter OpenRouter API key..."
|
||||
/>
|
||||
</FormField>
|
||||
<FormField
|
||||
label="OpenRouter Model"
|
||||
tooltip="Model identifier from OpenRouter (e.g., anthropic/claude-3.5-sonnet, google/gemini-2.0-flash-thinking-exp)"
|
||||
>
|
||||
<input
|
||||
type="text"
|
||||
value={formState.CLAUDE_MEM_OPENROUTER_MODEL || 'xiaomi/mimo-v2-flash:free'}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_OPENROUTER_MODEL', e.target.value)}
|
||||
placeholder="e.g., xiaomi/mimo-v2-flash:free"
|
||||
/>
|
||||
</FormField>
|
||||
<FormField
|
||||
label="Site URL (Optional)"
|
||||
tooltip="Your site URL for OpenRouter analytics (optional)"
|
||||
>
|
||||
<input
|
||||
type="text"
|
||||
value={formState.CLAUDE_MEM_OPENROUTER_SITE_URL || ''}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_OPENROUTER_SITE_URL', e.target.value)}
|
||||
placeholder="https://yoursite.com"
|
||||
/>
|
||||
</FormField>
|
||||
<FormField
|
||||
label="App Name (Optional)"
|
||||
tooltip="Your app name for OpenRouter analytics (optional)"
|
||||
>
|
||||
<input
|
||||
type="text"
|
||||
value={formState.CLAUDE_MEM_OPENROUTER_APP_NAME || 'claude-mem'}
|
||||
onChange={(e) => updateSetting('CLAUDE_MEM_OPENROUTER_APP_NAME', e.target.value)}
|
||||
placeholder="claude-mem"
|
||||
/>
|
||||
</FormField>
|
||||
</>
|
||||
)}
|
||||
|
||||
<FormField
|
||||
label="Worker Port"
|
||||
tooltip="Port for the background worker service"
|
||||
@@ -473,6 +561,20 @@ export function ContextSettingsModal({
|
||||
</CollapsibleSection>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Footer with Save button */}
|
||||
<div className="modal-footer">
|
||||
<div className="save-status">
|
||||
{saveStatus && <span className={saveStatus.includes('✓') ? 'success' : saveStatus.includes('✗') ? 'error' : ''}>{saveStatus}</span>}
|
||||
</div>
|
||||
<button
|
||||
className="save-btn"
|
||||
onClick={handleSave}
|
||||
disabled={isSaving}
|
||||
>
|
||||
{isSaving ? 'Saving...' : 'Save'}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
|
||||
@@ -2,6 +2,7 @@ import React from 'react';
|
||||
import { ThemeToggle } from './ThemeToggle';
|
||||
import { ThemePreference } from '../hooks/useTheme';
|
||||
import { GitHubStarsButton } from './GitHubStarsButton';
|
||||
import { useSpinningFavicon } from '../hooks/useSpinningFavicon';
|
||||
|
||||
interface HeaderProps {
|
||||
isConnected: boolean;
|
||||
@@ -26,6 +27,8 @@ export function Header({
|
||||
onThemeChange,
|
||||
onContextPreviewToggle
|
||||
}: HeaderProps) {
|
||||
useSpinningFavicon(isProcessing);
|
||||
|
||||
return (
|
||||
<div className="header">
|
||||
<h1>
|
||||
|
||||
@@ -8,6 +8,16 @@ export const DEFAULT_SETTINGS = {
|
||||
CLAUDE_MEM_WORKER_PORT: '37777',
|
||||
CLAUDE_MEM_WORKER_HOST: '127.0.0.1',
|
||||
|
||||
// AI Provider Configuration
|
||||
CLAUDE_MEM_PROVIDER: 'claude',
|
||||
CLAUDE_MEM_GEMINI_API_KEY: '',
|
||||
CLAUDE_MEM_GEMINI_MODEL: 'gemini-2.5-flash-lite',
|
||||
CLAUDE_MEM_OPENROUTER_API_KEY: '',
|
||||
CLAUDE_MEM_OPENROUTER_MODEL: 'xiaomi/mimo-v2-flash:free',
|
||||
CLAUDE_MEM_OPENROUTER_SITE_URL: '',
|
||||
CLAUDE_MEM_OPENROUTER_APP_NAME: 'claude-mem',
|
||||
CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED: 'true',
|
||||
|
||||
// Token Economics (all true for backwards compatibility)
|
||||
CLAUDE_MEM_CONTEXT_SHOW_READ_TOKENS: 'true',
|
||||
CLAUDE_MEM_CONTEXT_SHOW_WORK_TOKENS: 'true',
|
||||
|
||||
@@ -20,6 +20,18 @@ export function useSettings() {
|
||||
CLAUDE_MEM_WORKER_PORT: data.CLAUDE_MEM_WORKER_PORT || DEFAULT_SETTINGS.CLAUDE_MEM_WORKER_PORT,
|
||||
CLAUDE_MEM_WORKER_HOST: data.CLAUDE_MEM_WORKER_HOST || DEFAULT_SETTINGS.CLAUDE_MEM_WORKER_HOST,
|
||||
|
||||
// AI Provider Configuration
|
||||
CLAUDE_MEM_PROVIDER: data.CLAUDE_MEM_PROVIDER || DEFAULT_SETTINGS.CLAUDE_MEM_PROVIDER,
|
||||
CLAUDE_MEM_GEMINI_API_KEY: data.CLAUDE_MEM_GEMINI_API_KEY || DEFAULT_SETTINGS.CLAUDE_MEM_GEMINI_API_KEY,
|
||||
CLAUDE_MEM_GEMINI_MODEL: data.CLAUDE_MEM_GEMINI_MODEL || DEFAULT_SETTINGS.CLAUDE_MEM_GEMINI_MODEL,
|
||||
CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED: data.CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED || DEFAULT_SETTINGS.CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED,
|
||||
|
||||
// OpenRouter Configuration
|
||||
CLAUDE_MEM_OPENROUTER_API_KEY: data.CLAUDE_MEM_OPENROUTER_API_KEY || DEFAULT_SETTINGS.CLAUDE_MEM_OPENROUTER_API_KEY,
|
||||
CLAUDE_MEM_OPENROUTER_MODEL: data.CLAUDE_MEM_OPENROUTER_MODEL || DEFAULT_SETTINGS.CLAUDE_MEM_OPENROUTER_MODEL,
|
||||
CLAUDE_MEM_OPENROUTER_SITE_URL: data.CLAUDE_MEM_OPENROUTER_SITE_URL || DEFAULT_SETTINGS.CLAUDE_MEM_OPENROUTER_SITE_URL,
|
||||
CLAUDE_MEM_OPENROUTER_APP_NAME: data.CLAUDE_MEM_OPENROUTER_APP_NAME || DEFAULT_SETTINGS.CLAUDE_MEM_OPENROUTER_APP_NAME,
|
||||
|
||||
// Token Economics Display
|
||||
CLAUDE_MEM_CONTEXT_SHOW_READ_TOKENS: data.CLAUDE_MEM_CONTEXT_SHOW_READ_TOKENS || DEFAULT_SETTINGS.CLAUDE_MEM_CONTEXT_SHOW_READ_TOKENS,
|
||||
CLAUDE_MEM_CONTEXT_SHOW_WORK_TOKENS: data.CLAUDE_MEM_CONTEXT_SHOW_WORK_TOKENS || DEFAULT_SETTINGS.CLAUDE_MEM_CONTEXT_SHOW_WORK_TOKENS,
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
import { useEffect, useRef } from 'react';
|
||||
|
||||
/**
|
||||
* Hook that makes the browser tab favicon spin when isProcessing is true.
|
||||
* Uses canvas to rotate the logo image and dynamically update the favicon.
|
||||
*/
|
||||
export function useSpinningFavicon(isProcessing: boolean) {
|
||||
const animationRef = useRef<number | null>(null);
|
||||
const canvasRef = useRef<HTMLCanvasElement | null>(null);
|
||||
const imageRef = useRef<HTMLImageElement | null>(null);
|
||||
const rotationRef = useRef(0);
|
||||
const originalFaviconRef = useRef<string | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
// Create canvas once
|
||||
if (!canvasRef.current) {
|
||||
canvasRef.current = document.createElement('canvas');
|
||||
canvasRef.current.width = 32;
|
||||
canvasRef.current.height = 32;
|
||||
}
|
||||
|
||||
// Load image once
|
||||
if (!imageRef.current) {
|
||||
imageRef.current = new Image();
|
||||
imageRef.current.src = 'claude-mem-logomark.webp';
|
||||
}
|
||||
|
||||
// Store original favicon
|
||||
if (!originalFaviconRef.current) {
|
||||
const link = document.querySelector<HTMLLinkElement>('link[rel="icon"]');
|
||||
if (link) {
|
||||
originalFaviconRef.current = link.href;
|
||||
}
|
||||
}
|
||||
|
||||
const canvas = canvasRef.current;
|
||||
const ctx = canvas.getContext('2d');
|
||||
const image = imageRef.current;
|
||||
|
||||
if (!ctx) return;
|
||||
|
||||
const updateFavicon = (dataUrl: string) => {
|
||||
let link = document.querySelector<HTMLLinkElement>('link[rel="icon"]');
|
||||
if (!link) {
|
||||
link = document.createElement('link');
|
||||
link.rel = 'icon';
|
||||
document.head.appendChild(link);
|
||||
}
|
||||
link.href = dataUrl;
|
||||
};
|
||||
|
||||
const animate = () => {
|
||||
if (!image.complete) {
|
||||
animationRef.current = requestAnimationFrame(animate);
|
||||
return;
|
||||
}
|
||||
|
||||
// Rotate by ~4 degrees per frame (matches 1.5s for full rotation at 60fps)
|
||||
rotationRef.current += (2 * Math.PI) / 90;
|
||||
|
||||
ctx.clearRect(0, 0, 32, 32);
|
||||
ctx.save();
|
||||
ctx.translate(16, 16);
|
||||
ctx.rotate(rotationRef.current);
|
||||
ctx.drawImage(image, -16, -16, 32, 32);
|
||||
ctx.restore();
|
||||
|
||||
updateFavicon(canvas.toDataURL('image/png'));
|
||||
animationRef.current = requestAnimationFrame(animate);
|
||||
};
|
||||
|
||||
if (isProcessing) {
|
||||
rotationRef.current = 0;
|
||||
animate();
|
||||
} else {
|
||||
// Stop animation and restore original favicon
|
||||
if (animationRef.current) {
|
||||
cancelAnimationFrame(animationRef.current);
|
||||
animationRef.current = null;
|
||||
}
|
||||
if (originalFaviconRef.current) {
|
||||
updateFavicon(originalFaviconRef.current);
|
||||
}
|
||||
}
|
||||
|
||||
return () => {
|
||||
if (animationRef.current) {
|
||||
cancelAnimationFrame(animationRef.current);
|
||||
animationRef.current = null;
|
||||
}
|
||||
};
|
||||
}, [isProcessing]);
|
||||
}
|
||||
@@ -60,6 +60,16 @@ export interface Settings {
|
||||
CLAUDE_MEM_WORKER_PORT: string;
|
||||
CLAUDE_MEM_WORKER_HOST: string;
|
||||
|
||||
// AI Provider Configuration
|
||||
CLAUDE_MEM_PROVIDER?: string; // 'claude' | 'gemini' | 'openrouter'
|
||||
CLAUDE_MEM_GEMINI_API_KEY?: string;
|
||||
CLAUDE_MEM_GEMINI_MODEL?: string; // 'gemini-2.5-flash-lite' | 'gemini-2.5-flash' | 'gemini-3-flash'
|
||||
CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED?: string; // 'true' | 'false'
|
||||
CLAUDE_MEM_OPENROUTER_API_KEY?: string;
|
||||
CLAUDE_MEM_OPENROUTER_MODEL?: string;
|
||||
CLAUDE_MEM_OPENROUTER_SITE_URL?: string;
|
||||
CLAUDE_MEM_OPENROUTER_APP_NAME?: string;
|
||||
|
||||
// Token Economics Display
|
||||
CLAUDE_MEM_CONTEXT_SHOW_READ_TOKENS?: string;
|
||||
CLAUDE_MEM_CONTEXT_SHOW_WORK_TOKENS?: string;
|
||||
|
||||
@@ -31,7 +31,7 @@ export function getWorkerRestartInstructions(
|
||||
let message = `${prefix}${portInfo}\n\n`;
|
||||
message += `To restart the worker:\n`;
|
||||
message += `1. Exit Claude Code completely\n`;
|
||||
message += `2. Run: claude-mem restart\n`;
|
||||
message += `2. Run: npm run worker:restart\n`;
|
||||
message += `3. Restart Claude Code`;
|
||||
|
||||
if (includeSkillFallback) {
|
||||
|
||||
+51
-7
@@ -4,6 +4,8 @@
|
||||
*/
|
||||
|
||||
import { SettingsDefaultsManager } from '../shared/SettingsDefaultsManager.js';
|
||||
import { appendFileSync, existsSync, mkdirSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
|
||||
export enum LogLevel {
|
||||
DEBUG = 0,
|
||||
@@ -25,19 +27,55 @@ interface LogContext {
|
||||
class Logger {
|
||||
private level: LogLevel | null = null;
|
||||
private useColor: boolean;
|
||||
private logFilePath: string | null = null;
|
||||
|
||||
constructor() {
|
||||
// Disable colors when output is not a TTY (e.g., PM2 logs)
|
||||
this.useColor = process.stdout.isTTY ?? false;
|
||||
this.initializeLogFile();
|
||||
}
|
||||
|
||||
/**
|
||||
* Lazy-load log level from settings (breaks circular dependency with SettingsDefaultsManager)
|
||||
* Initialize log file path and ensure directory exists
|
||||
*/
|
||||
private initializeLogFile(): void {
|
||||
try {
|
||||
// Get data directory from settings
|
||||
const dataDir = SettingsDefaultsManager.get('CLAUDE_MEM_DATA_DIR');
|
||||
const logsDir = join(dataDir, 'logs');
|
||||
|
||||
// Ensure logs directory exists
|
||||
if (!existsSync(logsDir)) {
|
||||
mkdirSync(logsDir, { recursive: true });
|
||||
}
|
||||
|
||||
// Create log file path with date
|
||||
const date = new Date().toISOString().split('T')[0];
|
||||
this.logFilePath = join(logsDir, `claude-mem-${date}.log`);
|
||||
} catch (error) {
|
||||
// If log file initialization fails, just log to console
|
||||
console.error('[LOGGER] Failed to initialize log file:', error);
|
||||
this.logFilePath = null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Lazy-load log level from settings file (not hardcoded defaults!)
|
||||
*/
|
||||
private getLevel(): LogLevel {
|
||||
if (this.level === null) {
|
||||
const envLevel = SettingsDefaultsManager.get('CLAUDE_MEM_LOG_LEVEL').toUpperCase();
|
||||
this.level = LogLevel[envLevel as keyof typeof LogLevel] ?? LogLevel.INFO;
|
||||
try {
|
||||
// Load settings from file to get user's actual log level
|
||||
const dataDir = SettingsDefaultsManager.get('CLAUDE_MEM_DATA_DIR');
|
||||
const settingsPath = join(dataDir, 'settings.json');
|
||||
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
|
||||
const envLevel = settings.CLAUDE_MEM_LOG_LEVEL.toUpperCase();
|
||||
this.level = LogLevel[envLevel as keyof typeof LogLevel] ?? LogLevel.INFO;
|
||||
} catch (error) {
|
||||
// Fallback to INFO if settings can't be loaded
|
||||
console.error('[LOGGER] Failed to load settings, using INFO level:', error);
|
||||
this.level = LogLevel.INFO;
|
||||
}
|
||||
}
|
||||
return this.level;
|
||||
}
|
||||
@@ -219,11 +257,17 @@ class Logger {
|
||||
|
||||
const logLine = `[${timestamp}] [${levelStr}] [${componentStr}] ${correlationStr}${message}${contextStr}${dataStr}`;
|
||||
|
||||
// Output to appropriate stream
|
||||
if (level === LogLevel.ERROR) {
|
||||
console.error(logLine);
|
||||
// Output to log file ONLY (worker runs in background, console is useless)
|
||||
if (this.logFilePath) {
|
||||
try {
|
||||
appendFileSync(this.logFilePath, logLine + '\n', 'utf8');
|
||||
} catch (error) {
|
||||
// If file write fails, write to stderr as last resort
|
||||
process.stderr.write(`[LOGGER] Failed to write to log file: ${error}\n`);
|
||||
}
|
||||
} else {
|
||||
console.log(logLine);
|
||||
// If no log file available, write to stderr as fallback
|
||||
process.stderr.write(logLine + '\n');
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,298 @@
|
||||
import { describe, it, expect, beforeEach, afterEach, spyOn, mock } from 'bun:test';
|
||||
import { GeminiAgent } from '../src/services/worker/GeminiAgent';
|
||||
import { DatabaseManager } from '../src/services/worker/DatabaseManager';
|
||||
import { SessionManager } from '../src/services/worker/SessionManager';
|
||||
import { ModeManager } from '../src/services/worker/domain/ModeManager';
|
||||
import { SettingsDefaultsManager } from '../src/shared/SettingsDefaultsManager';
|
||||
|
||||
let billingEnabled = 'true';
|
||||
|
||||
// Mock SettingsDefaultsManager
|
||||
mock.module('../src/shared/SettingsDefaultsManager', () => ({
|
||||
SettingsDefaultsManager: {
|
||||
loadFromFile: () => ({
|
||||
CLAUDE_MEM_GEMINI_API_KEY: 'test-api-key',
|
||||
CLAUDE_MEM_GEMINI_MODEL: 'gemini-2.5-flash-lite',
|
||||
CLAUDE_MEM_GEMINI_BILLING_ENABLED: billingEnabled
|
||||
}),
|
||||
get: (key: string) => {
|
||||
if (key === 'CLAUDE_MEM_LOG_LEVEL') return 'INFO';
|
||||
return '';
|
||||
}
|
||||
}
|
||||
}));
|
||||
|
||||
// Mock ModeManager
|
||||
const mockMode = {
|
||||
name: 'code',
|
||||
prompts: {
|
||||
init: 'init prompt',
|
||||
observation: 'obs prompt',
|
||||
summary: 'summary prompt'
|
||||
},
|
||||
observation_types: [{ id: 'discovery' }, { id: 'bugfix' }],
|
||||
observation_concepts: []
|
||||
};
|
||||
|
||||
mock.module('../src/services/domain/ModeManager', () => ({
|
||||
ModeManager: {
|
||||
getInstance: () => ({
|
||||
getActiveMode: () => mockMode
|
||||
})
|
||||
}
|
||||
}));
|
||||
|
||||
describe('GeminiAgent', () => {
|
||||
let agent: GeminiAgent;
|
||||
let originalFetch: typeof global.fetch;
|
||||
|
||||
// Mocks
|
||||
let mockStoreObservation: any;
|
||||
let mockStoreSummary: any;
|
||||
let mockMarkSessionCompleted: any;
|
||||
let mockSyncObservation: any;
|
||||
let mockSyncSummary: any;
|
||||
let mockMarkProcessed: any;
|
||||
let mockCleanupProcessed: any;
|
||||
let mockResetStuckMessages: any;
|
||||
let mockDbManager: DatabaseManager;
|
||||
let mockSessionManager: SessionManager;
|
||||
|
||||
beforeEach(() => {
|
||||
// Reset billing for each test default
|
||||
billingEnabled = 'true';
|
||||
|
||||
// Initialize mocks
|
||||
mockStoreObservation = mock(() => ({ id: 1, createdAtEpoch: Date.now() }));
|
||||
mockStoreSummary = mock(() => ({ id: 1, createdAtEpoch: Date.now() }));
|
||||
mockMarkSessionCompleted = mock(() => {});
|
||||
mockSyncObservation = mock(() => Promise.resolve());
|
||||
mockSyncSummary = mock(() => Promise.resolve());
|
||||
mockMarkProcessed = mock(() => {});
|
||||
mockCleanupProcessed = mock(() => 0);
|
||||
mockResetStuckMessages = mock(() => 0);
|
||||
|
||||
const mockSessionStore = {
|
||||
storeObservation: mockStoreObservation,
|
||||
storeSummary: mockStoreSummary,
|
||||
markSessionCompleted: mockMarkSessionCompleted
|
||||
};
|
||||
|
||||
const mockChromaSync = {
|
||||
syncObservation: mockSyncObservation,
|
||||
syncSummary: mockSyncSummary
|
||||
};
|
||||
|
||||
mockDbManager = {
|
||||
getSessionStore: () => mockSessionStore,
|
||||
getChromaSync: () => mockChromaSync
|
||||
} as unknown as DatabaseManager;
|
||||
|
||||
const mockPendingMessageStore = {
|
||||
markProcessed: mockMarkProcessed,
|
||||
cleanupProcessed: mockCleanupProcessed,
|
||||
resetStuckMessages: mockResetStuckMessages
|
||||
};
|
||||
|
||||
mockSessionManager = {
|
||||
getMessageIterator: async function* () { yield* []; },
|
||||
getPendingMessageStore: () => mockPendingMessageStore
|
||||
} as unknown as SessionManager;
|
||||
|
||||
agent = new GeminiAgent(mockDbManager, mockSessionManager);
|
||||
originalFetch = global.fetch;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
global.fetch = originalFetch;
|
||||
mock.restore();
|
||||
});
|
||||
|
||||
it('should initialize with correct config', async () => {
|
||||
const session = {
|
||||
sessionDbId: 1,
|
||||
claudeSessionId: 'test-session',
|
||||
sdkSessionId: 'test-sdk',
|
||||
project: 'test-project',
|
||||
userPrompt: 'test prompt',
|
||||
conversationHistory: [],
|
||||
lastPromptNumber: 1,
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0,
|
||||
pendingProcessingIds: new Set(),
|
||||
startTime: Date.now()
|
||||
} as any;
|
||||
|
||||
global.fetch = mock(() => Promise.resolve(new Response(JSON.stringify({
|
||||
candidates: [{
|
||||
content: {
|
||||
parts: [{ text: '<observation><type>discovery</type><title>Test</title></observation>' }]
|
||||
}
|
||||
}],
|
||||
usageMetadata: { totalTokenCount: 100 }
|
||||
}))));
|
||||
|
||||
await agent.startSession(session);
|
||||
|
||||
expect(global.fetch).toHaveBeenCalledTimes(1);
|
||||
const url = (global.fetch as any).mock.calls[0][0];
|
||||
expect(url).toContain('https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash-lite:generateContent');
|
||||
expect(url).toContain('key=test-api-key');
|
||||
});
|
||||
|
||||
it('should handle multi-turn conversation', async () => {
|
||||
const session = {
|
||||
sessionDbId: 1,
|
||||
claudeSessionId: 'test-session',
|
||||
sdkSessionId: 'test-sdk',
|
||||
project: 'test-project',
|
||||
userPrompt: 'test prompt',
|
||||
conversationHistory: [{ role: 'user', content: 'prev context' }, { role: 'assistant', content: 'prev response' }],
|
||||
lastPromptNumber: 2,
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0,
|
||||
pendingProcessingIds: new Set(),
|
||||
startTime: Date.now()
|
||||
} as any;
|
||||
|
||||
global.fetch = mock(() => Promise.resolve(new Response(JSON.stringify({
|
||||
candidates: [{ content: { parts: [{ text: 'response' }] } }]
|
||||
}))));
|
||||
|
||||
await agent.startSession(session);
|
||||
|
||||
const body = JSON.parse((global.fetch as any).mock.calls[0][1].body);
|
||||
expect(body.contents).toHaveLength(3);
|
||||
expect(body.contents[0].role).toBe('user');
|
||||
expect(body.contents[1].role).toBe('model');
|
||||
expect(body.contents[2].role).toBe('user');
|
||||
});
|
||||
|
||||
it('should process observations and store them', async () => {
|
||||
const session = {
|
||||
sessionDbId: 1,
|
||||
claudeSessionId: 'test-session',
|
||||
sdkSessionId: 'test-sdk',
|
||||
project: 'test-project',
|
||||
userPrompt: 'test prompt',
|
||||
conversationHistory: [],
|
||||
lastPromptNumber: 1,
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0,
|
||||
pendingProcessingIds: new Set(),
|
||||
startTime: Date.now()
|
||||
} as any;
|
||||
|
||||
const observationXml = `
|
||||
<observation>
|
||||
<type>discovery</type>
|
||||
<title>Found bug</title>
|
||||
<subtitle>Null pointer</subtitle>
|
||||
<narrative>Found a null pointer in the code</narrative>
|
||||
<facts><fact>Null check missing</fact></facts>
|
||||
<concepts><concept>bug</concept></concepts>
|
||||
<files_read><file>src/main.ts</file></files_read>
|
||||
<files_modified></files_modified>
|
||||
</observation>
|
||||
`;
|
||||
|
||||
global.fetch = mock(() => Promise.resolve(new Response(JSON.stringify({
|
||||
candidates: [{ content: { parts: [{ text: observationXml }] } }],
|
||||
usageMetadata: { totalTokenCount: 50 }
|
||||
}))));
|
||||
|
||||
await agent.startSession(session);
|
||||
|
||||
expect(mockStoreObservation).toHaveBeenCalled();
|
||||
expect(mockSyncObservation).toHaveBeenCalled();
|
||||
expect(session.cumulativeInputTokens).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('should fallback to Claude on rate limit error', async () => {
|
||||
const session = {
|
||||
sessionDbId: 1,
|
||||
claudeSessionId: 'test-session',
|
||||
sdkSessionId: 'test-sdk',
|
||||
project: 'test-project',
|
||||
userPrompt: 'test prompt',
|
||||
conversationHistory: [],
|
||||
lastPromptNumber: 1,
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0,
|
||||
pendingProcessingIds: new Set(),
|
||||
startTime: Date.now()
|
||||
} as any;
|
||||
|
||||
global.fetch = mock(() => Promise.resolve(new Response('Resource has been exhausted (e.g. check quota).', { status: 429 })));
|
||||
|
||||
const fallbackAgent = {
|
||||
startSession: mock(() => Promise.resolve())
|
||||
};
|
||||
agent.setFallbackAgent(fallbackAgent);
|
||||
|
||||
await agent.startSession(session);
|
||||
|
||||
expect(fallbackAgent.startSession).toHaveBeenCalledWith(session, undefined);
|
||||
expect(mockResetStuckMessages).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should NOT fallback on other errors', async () => {
|
||||
const session = {
|
||||
sessionDbId: 1,
|
||||
claudeSessionId: 'test-session',
|
||||
sdkSessionId: 'test-sdk',
|
||||
project: 'test-project',
|
||||
userPrompt: 'test prompt',
|
||||
conversationHistory: [],
|
||||
lastPromptNumber: 1,
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0,
|
||||
pendingProcessingIds: new Set(),
|
||||
startTime: Date.now()
|
||||
} as any;
|
||||
|
||||
global.fetch = mock(() => Promise.resolve(new Response('Invalid argument', { status: 400 })));
|
||||
|
||||
const fallbackAgent = {
|
||||
startSession: mock(() => Promise.resolve())
|
||||
};
|
||||
agent.setFallbackAgent(fallbackAgent);
|
||||
|
||||
expect(agent.startSession(session)).rejects.toThrow('Gemini API error: 400 - Invalid argument');
|
||||
expect(fallbackAgent.startSession).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should respect rate limits when billing disabled', async () => {
|
||||
billingEnabled = 'false';
|
||||
const originalSetTimeout = global.setTimeout;
|
||||
const mockSetTimeout = mock((cb: any) => cb());
|
||||
global.setTimeout = mockSetTimeout as any;
|
||||
|
||||
try {
|
||||
const session = {
|
||||
sessionDbId: 1,
|
||||
claudeSessionId: 'test-session',
|
||||
sdkSessionId: 'test-sdk',
|
||||
project: 'test-project',
|
||||
userPrompt: 'test prompt',
|
||||
conversationHistory: [],
|
||||
lastPromptNumber: 1,
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0,
|
||||
pendingProcessingIds: new Set(),
|
||||
startTime: Date.now()
|
||||
} as any;
|
||||
|
||||
global.fetch = mock(() => Promise.resolve(new Response(JSON.stringify({
|
||||
candidates: [{ content: { parts: [{ text: 'ok' }] } }]
|
||||
}))));
|
||||
|
||||
await agent.startSession(session);
|
||||
await agent.startSession(session);
|
||||
|
||||
expect(mockSetTimeout).toHaveBeenCalled();
|
||||
} finally {
|
||||
global.setTimeout = originalSetTimeout;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,100 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { HOOK_TIMEOUTS, HOOK_EXIT_CODES, getTimeout } from '../src/shared/hook-constants.js';
|
||||
|
||||
describe('hook-constants', () => {
|
||||
const originalPlatform = process.platform;
|
||||
|
||||
afterEach(() => {
|
||||
// Restore original platform after each test
|
||||
Object.defineProperty(process, 'platform', {
|
||||
value: originalPlatform,
|
||||
writable: true,
|
||||
configurable: true
|
||||
});
|
||||
});
|
||||
|
||||
describe('HOOK_TIMEOUTS', () => {
|
||||
it('should define DEFAULT timeout', () => {
|
||||
expect(HOOK_TIMEOUTS.DEFAULT).toBe(300000);
|
||||
});
|
||||
|
||||
it('should define HEALTH_CHECK timeout', () => {
|
||||
expect(HOOK_TIMEOUTS.HEALTH_CHECK).toBe(30000);
|
||||
});
|
||||
|
||||
it('should define WORKER_STARTUP_WAIT', () => {
|
||||
expect(HOOK_TIMEOUTS.WORKER_STARTUP_WAIT).toBe(1000);
|
||||
});
|
||||
|
||||
it('should define WORKER_STARTUP_RETRIES', () => {
|
||||
expect(HOOK_TIMEOUTS.WORKER_STARTUP_RETRIES).toBe(300);
|
||||
});
|
||||
|
||||
it('should define PRE_RESTART_SETTLE_DELAY', () => {
|
||||
expect(HOOK_TIMEOUTS.PRE_RESTART_SETTLE_DELAY).toBe(2000);
|
||||
});
|
||||
|
||||
it('should define WINDOWS_MULTIPLIER', () => {
|
||||
expect(HOOK_TIMEOUTS.WINDOWS_MULTIPLIER).toBe(1.5);
|
||||
});
|
||||
});
|
||||
|
||||
describe('HOOK_EXIT_CODES', () => {
|
||||
it('should define SUCCESS exit code', () => {
|
||||
expect(HOOK_EXIT_CODES.SUCCESS).toBe(0);
|
||||
});
|
||||
|
||||
it('should define FAILURE exit code', () => {
|
||||
expect(HOOK_EXIT_CODES.FAILURE).toBe(1);
|
||||
});
|
||||
|
||||
it('should define USER_MESSAGE_ONLY exit code', () => {
|
||||
expect(HOOK_EXIT_CODES.USER_MESSAGE_ONLY).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
describe('getTimeout', () => {
|
||||
it('should return base timeout on non-Windows platforms', () => {
|
||||
Object.defineProperty(process, 'platform', {
|
||||
value: 'darwin',
|
||||
writable: true,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
expect(getTimeout(1000)).toBe(1000);
|
||||
expect(getTimeout(5000)).toBe(5000);
|
||||
});
|
||||
|
||||
it('should apply Windows multiplier on Windows platform', () => {
|
||||
Object.defineProperty(process, 'platform', {
|
||||
value: 'win32',
|
||||
writable: true,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
expect(getTimeout(1000)).toBe(1500);
|
||||
expect(getTimeout(2000)).toBe(3000);
|
||||
});
|
||||
|
||||
it('should round Windows timeout to nearest integer', () => {
|
||||
Object.defineProperty(process, 'platform', {
|
||||
value: 'win32',
|
||||
writable: true,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
// 333 * 1.5 = 499.5, should round to 500
|
||||
expect(getTimeout(333)).toBe(500);
|
||||
});
|
||||
|
||||
it('should return base timeout on Linux', () => {
|
||||
Object.defineProperty(process, 'platform', {
|
||||
value: 'linux',
|
||||
writable: true,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
expect(getTimeout(1000)).toBe(1000);
|
||||
});
|
||||
});
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user