Files
claude-mem/PHASED-EXECUTION-PLAN-SIMPLE.md
T
Alex Newman 266c746d50 feat: Fix observation timestamps, refactor session management, and enhance worker reliability (#437)
* Refactor worker version checks and increase timeout settings

- Updated the default hook timeout from 5000ms to 120000ms for improved stability.
- Modified the worker version check to log a warning instead of restarting the worker on version mismatch.
- Removed legacy PM2 cleanup and worker start logic, simplifying the ensureWorkerRunning function.
- Enhanced polling mechanism for worker readiness with increased retries and reduced interval.

* feat: implement worker queue polling to ensure processing completion before proceeding

* refactor: change worker command from start to restart in hooks configuration

* refactor: remove session management complexity

- Simplify createSDKSession to pure INSERT OR IGNORE
- Remove auto-create logic from storeObservation/storeSummary
- Delete 11 unused session management methods
- Derive prompt_number from user_prompts count
- Keep sdk_sessions table schema unchanged for compatibility

* refactor: simplify session management by removing unused methods and auto-creation logic

* Refactor session prompt number retrieval in SessionRoutes

- Updated the method of obtaining the prompt number from the session.
- Replaced `store.getPromptCounter(sessionDbId)` with `store.getPromptNumberFromUserPrompts(claudeSessionId)` for better clarity and accuracy.
- Adjusted the logic for incrementing the prompt number to derive it from the user prompts count instead of directly incrementing a counter.

* refactor: replace getPromptCounter with getPromptNumberFromUserPrompts in SessionManager

Phase 7 of session management simplification. Updates SessionManager to derive
prompt numbers from user_prompts table count instead of using the deprecated
prompt_counter column.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: simplify SessionCompletionHandler to use direct SQL query

Phase 8: Remove call to findActiveSDKSession() and replace with direct
database query in SessionCompletionHandler.completeByClaudeId().

This removes dependency on the deleted findActiveSDKSession() method
and simplifies the code by using a straightforward SELECT query.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: remove markSessionCompleted call from SDKAgent

- Delete call to markSessionCompleted() in SDKAgent.ts
- Session status is no longer tracked or updated
- Part of phase 9: simplifying session management

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: remove markSessionComplete method (Phase 10)

- Deleted markSessionComplete() method from DatabaseManager
- Removed markSessionComplete call from SessionCompletionHandler
- Session completion status no longer tracked in database
- Part of session management simplification effort

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: replace deleted updateSDKSessionId calls in import script (Phase 11)

- Replace updateSDKSessionId() calls with direct SQL UPDATE statements
- Method was deleted in Phase 3 as part of session management simplification
- Import script now uses direct database access consistently

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* test: add validation for SQL updates in sdk_sessions table

* refactor: enhance worker-cli to support manual and automated runs

* Remove cleanup hook and associated session completion logic

- Deleted the cleanup-hook implementation from the hooks directory.
- Removed the session completion endpoint that was used by the cleanup hook.
- Updated the SessionCompletionHandler to eliminate the completeByClaudeId method and its dependencies.
- Adjusted the SessionRoutes to reflect the removal of the session completion route.

* fix: update worker-cli command to use bun for consistency

* feat: Implement timestamp fix for observations and enhance processing logic

- Added `earliestPendingTimestamp` to `ActiveSession` to track the original timestamp of the earliest pending message.
- Updated `SDKAgent` to capture and utilize the earliest pending timestamp during response processing.
- Modified `SessionManager` to track the earliest timestamp when yielding messages.
- Created scripts for fixing corrupted timestamps, validating fixes, and investigating timestamp issues.
- Verified that all corrupted observations have been repaired and logic for future processing is sound.
- Ensured orphan processing can be safely re-enabled after validation.

* feat: Enhance SessionStore to support custom database paths and add timestamp fields for observations and summaries

* Refactor pending queue processing and add management endpoints

- Disabled automatic recovery of orphaned queues on startup; users must now use the new /api/pending-queue/process endpoint.
- Updated processOrphanedQueues method to processPendingQueues with improved session handling and return detailed results.
- Added new API endpoints for managing pending queues: GET /api/pending-queue and POST /api/pending-queue/process.
- Introduced a new script (check-pending-queue.ts) for checking and processing pending observation queues interactively or automatically.
- Enhanced logging and error handling for better monitoring of session processing.

* updated agent sdk

* feat: Add manual recovery guide and queue management endpoints to documentation

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-25 15:36:46 -05:00

12 KiB

Phased Execution Plan: Remove Session Management Logic

Goal: Delete problematic session management code while keeping schema unchanged

Approach: Stop using certain columns/methods, but don't change the database structure


PHASE 1: Simplify createSDKSession()

File: src/services/sqlite/SessionStore.ts

Task: Replace createSDKSession() with a pure INSERT OR IGNORE version

Current code (~line 1142-1178):

  • Does INSERT OR IGNORE
  • Then tries to UPDATE project/user_prompt if they changed
  • Complex logic

New code:

createSDKSession(claudeSessionId: string, project: string, userPrompt: string): number {
  const now = new Date();
  const nowEpoch = now.getTime();

  // Pure INSERT OR IGNORE - no updates, no complexity
  this.db.prepare(`
    INSERT OR IGNORE INTO sdk_sessions
    (claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
    VALUES (?, ?, ?, ?, ?, ?, 'active')
  `).run(claudeSessionId, claudeSessionId, project, userPrompt, now.toISOString(), nowEpoch);

  // Return existing or new ID
  const row = this.db.prepare('SELECT id FROM sdk_sessions WHERE claude_session_id = ?')
    .get(claudeSessionId) as { id: number };
  return row.id;
}

Verification:

grep -A20 "createSDKSession(" src/services/sqlite/SessionStore.ts | head -25
# Should see simple INSERT OR IGNORE, no UPDATE logic

Output: Simplified createSDKSession method


PHASE 2: Add getPromptNumberFromUserPrompts() Helper

File: src/services/sqlite/SessionStore.ts

Task: Add a new helper method to derive prompt number from user_prompts table

Add this method:

/**
 * Get current prompt number by counting user_prompts for this session
 * Replaces the prompt_counter column which is no longer maintained
 */
getPromptNumberFromUserPrompts(claudeSessionId: string): number {
  const result = this.db.prepare(`
    SELECT COUNT(*) as count FROM user_prompts WHERE claude_session_id = ?
  `).get(claudeSessionId) as { count: number };
  return result.count;
}

Verification:

grep -n "getPromptNumberFromUserPrompts" src/services/sqlite/SessionStore.ts
# Should find the new method

Output: New helper method added


PHASE 3: Delete Unused Session Management Methods

File: src/services/sqlite/SessionStore.ts

Task: Delete these 11 methods entirely:

  1. updateSDKSessionId() (~line 1185-1205)
  2. findActiveSDKSession() (~line 1043-1057)
  3. findAnySDKSession() (~line 1062-1071)
  4. reactivateSession() (~line 1076-1084)
  5. incrementPromptCounter() (~line 1089-1103)
  6. getPromptCounter() (~line 1108-1114)
  7. setWorkerPort() (~line 1210-1218)
  8. getWorkerPort() (~line 1223-1233)
  9. markSessionCompleted() (~line 1419-1430)
  10. markSessionFailed() (~line 1435-1446)
  11. getSdkSessionsBySessionIds() (~line 1017-1038) - if unused

Verification:

# These should return no results after deletion
grep -n "incrementPromptCounter\|getPromptCounter\|setWorkerPort\|getWorkerPort" src/services/sqlite/SessionStore.ts
grep -n "markSessionCompleted\|markSessionFailed\|reactivateSession" src/services/sqlite/SessionStore.ts
grep -n "findActiveSDKSession\|findAnySDKSession\|updateSDKSessionId" src/services/sqlite/SessionStore.ts

Output: 11 methods deleted from SessionStore.ts


PHASE 4: Remove Auto-Create from storeObservation()

File: src/services/sqlite/SessionStore.ts

Task: Remove session auto-creation logic from storeObservation()

Find (~line 1291-1312): The block that checks if session exists and creates it if not

Delete this pattern:

// DELETE THIS BLOCK:
let sessionId = this.db.prepare(`SELECT id FROM sdk_sessions WHERE sdk_session_id = ?`).get(sdkSessionId);
if (!sessionId) {
  // auto-create session...
}

Keep: Just the INSERT INTO observations statement

The method should assume the session already exists. If it doesn't, the INSERT will fail with a foreign key error - which is correct behavior (means the hook is broken).

Verification:

grep -B5 -A15 "storeObservation(" src/services/sqlite/SessionStore.ts | grep -i "insert.*sdk_sessions"
# Should return nothing - no session creation in storeObservation

Output: storeObservation() no longer auto-creates sessions


PHASE 5: Remove Auto-Create from storeSummary()

File: src/services/sqlite/SessionStore.ts

Task: Remove session auto-creation logic from storeSummary()

Find (~line 1367-1388): Similar pattern to storeObservation

Delete the session existence check and auto-create block

Keep: Just the INSERT INTO session_summaries statement

Verification:

grep -B5 -A15 "storeSummary(" src/services/sqlite/SessionStore.ts | grep -i "insert.*sdk_sessions"
# Should return nothing - no session creation in storeSummary

Output: storeSummary() no longer auto-creates sessions


PHASE 6: Update SessionRoutes - Replace getPromptCounter

File: src/services/worker/http/routes/SessionRoutes.ts

Task: Replace calls to getPromptCounter() with getPromptNumberFromUserPrompts()

Find: All calls to store.getPromptCounter(sessionDbId)

Replace with: store.getPromptNumberFromUserPrompts(claudeSessionId)

Note: The new method takes claudeSessionId (string), not sessionDbId (number)

Verification:

grep -n "getPromptCounter" src/services/worker/http/routes/SessionRoutes.ts
# Should return nothing

grep -n "getPromptNumberFromUserPrompts" src/services/worker/http/routes/SessionRoutes.ts
# Should find the new calls

Output: SessionRoutes uses new prompt counting method


PHASE 7: Update SessionManager - Replace getPromptCounter

File: src/services/worker/SessionManager.ts

Task: Replace call to getPromptCounter() in initializeSession()

Find (~line 116): this.dbManager.getSessionStore().getPromptCounter(sessionDbId)

Replace with:

this.dbManager.getSessionStore().getPromptNumberFromUserPrompts(dbSession.claude_session_id)

Verification:

grep -n "getPromptCounter" src/services/worker/SessionManager.ts
# Should return nothing

Output: SessionManager uses new prompt counting method


PHASE 8: Update SessionCompletionHandler

File: src/services/worker/session/SessionCompletionHandler.ts

Task: Remove call to findActiveSDKSession() and simplify

Find: Call to store.findActiveSDKSession(claudeSessionId)

Replace: Use a simpler query or remove the lookup if not needed

If the method just needs to find the session to delete it, we can query sdk_sessions directly:

const session = store.db.prepare(
  'SELECT id FROM sdk_sessions WHERE claude_session_id = ?'
).get(claudeSessionId);

Verification:

grep -n "findActiveSDKSession" src/services/worker/session/SessionCompletionHandler.ts
# Should return nothing

Output: SessionCompletionHandler no longer uses deleted method


PHASE 9: Update SDKAgent - Remove markSessionCompleted

File: src/services/worker/SDKAgent.ts

Task: Remove call to markSessionCompleted()

Find (~line 148): this.dbManager.getSessionStore().markSessionCompleted(session.sessionDbId)

Action: Delete this line entirely. We no longer track session status.

Verification:

grep -n "markSessionCompleted\|markSessionFailed" src/services/worker/SDKAgent.ts
# Should return nothing

Output: SDKAgent no longer marks sessions as completed


PHASE 10: Update DatabaseManager - Remove markSessionComplete

File: src/services/worker/DatabaseManager.ts

Task: Remove the markSessionComplete() method

Find (~line 116-118): The markSessionComplete method

Action: Delete the entire method

Verification:

grep -n "markSessionComplete" src/services/worker/DatabaseManager.ts
# Should return nothing

Output: DatabaseManager no longer has session completion method


PHASE 11: Search for Any Remaining References

Task: Find and fix any remaining references to deleted methods

Search:

# Search for all deleted method names
grep -rn "incrementPromptCounter\|getPromptCounter\|setWorkerPort\|getWorkerPort" src/ --include="*.ts"
grep -rn "markSessionCompleted\|markSessionFailed\|reactivateSession" src/ --include="*.ts"
grep -rn "findActiveSDKSession\|findAnySDKSession\|updateSDKSessionId" src/ --include="*.ts"

Action: Fix any remaining references found

Output: No references to deleted methods remain


PHASE 12: Build and Test

Task: Verify everything compiles and works

Commands:

# Build
npm run build

# If build fails, fix TypeScript errors and rebuild

Verification:

  • Build completes without errors
  • No TypeScript errors about missing methods

Output: Clean build


PHASE 13: Integration Test

Task: Test the complete flow

Test steps:

  1. Start a new Claude Code session
  2. Submit a prompt (triggers new-hook → createSDKSession)
  3. Use some tools (triggers save-hook → storeObservation)
  4. End session (triggers summary-hook → storeSummary)

Verify in database:

sqlite3 ~/.claude-mem/claude-mem.db "SELECT id, claude_session_id, project, status FROM sdk_sessions ORDER BY id DESC LIMIT 3;"
sqlite3 ~/.claude-mem/claude-mem.db "SELECT id, sdk_session_id, type, title FROM observations ORDER BY id DESC LIMIT 5;"

Expected:

  • Sessions created with status='active' (never updated, that's fine)
  • Observations saved correctly
  • No errors in worker logs

Output: System works end-to-end


PHASE 14: Cleanup Plan Files

Task: Delete the planning files

rm PLAN-REMOVE-SESSION-MANAGEMENT.md
rm PHASED-EXECUTION-PLAN.md
rm PHASED-EXECUTION-PLAN-SIMPLE.md

Commit:

git add -A
git commit -m "refactor: remove session management complexity

- Simplify createSDKSession to pure INSERT OR IGNORE
- Remove auto-create logic from storeObservation/storeSummary
- Delete 11 unused session management methods
- Derive prompt_number from user_prompts count
- Keep sdk_sessions table schema unchanged for compatibility"

Output: Clean commit with simplified session handling


SUCCESS CRITERIA

After all phases complete, verify:

  • createSDKSession() is pure INSERT OR IGNORE (no updates)
  • storeObservation() has no session auto-create logic
  • storeSummary() has no session auto-create logic
  • 11 session management methods deleted
  • getPromptNumberFromUserPrompts() exists and is used
  • Build completes without errors
  • Normal flow works (prompt → tools → observations → summary)
  • No references to deleted methods in codebase
  • Database schema unchanged (no migration needed)

WHAT WE STOPPED USING (BUT DIDN'T DELETE FROM SCHEMA)

These columns in sdk_sessions are now dead:

Column Previously Now
status Updated to 'completed'/'failed' Always 'active', never updated
completed_at Set on session end Never set
completed_at_epoch Set on session end Never set
worker_port Tracked which worker Never set
prompt_counter Incremented per prompt Ignored, derived from user_prompts

These columns still work fine but we just don't write to them anymore. Existing data is preserved.


PHILOSOPHY

Before: Complex session lifecycle management with status tracking, port assignment, prompt counting, auto-creation fallbacks

After: Simple lookup table. INSERT OR IGNORE on first prompt. That's it.

If data is missing, it's a bug in the hook. Fail loudly, don't paper over it.