feat: Fix observation timestamps, refactor session management, and enhance worker reliability (#437)

* Refactor worker version checks and increase timeout settings - Updated the default hook timeout from 5000ms to 120000ms for improved stability. - Modified the worker version check to log a warning instead of restarting the worker on version mismatch. - Removed legacy PM2 cleanup and worker start logic, simplifying the ensureWorkerRunning function. - Enhanced polling mechanism for worker readiness with increased retries and reduced interval. * feat: implement worker queue polling to ensure processing completion before proceeding * refactor: change worker command from start to restart in hooks configuration * refactor: remove session management complexity - Simplify createSDKSession to pure INSERT OR IGNORE - Remove auto-create logic from storeObservation/storeSummary - Delete 11 unused session management methods - Derive prompt_number from user_prompts count - Keep sdk_sessions table schema unchanged for compatibility * refactor: simplify session management by removing unused methods and auto-creation logic * Refactor session prompt number retrieval in SessionRoutes - Updated the method of obtaining the prompt number from the session. - Replaced `store.getPromptCounter(sessionDbId)` with `store.getPromptNumberFromUserPrompts(claudeSessionId)` for better clarity and accuracy. - Adjusted the logic for incrementing the prompt number to derive it from the user prompts count instead of directly incrementing a counter. * refactor: replace getPromptCounter with getPromptNumberFromUserPrompts in SessionManager Phase 7 of session management simplification. Updates SessionManager to derive prompt numbers from user_prompts table count instead of using the deprecated prompt_counter column. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: simplify SessionCompletionHandler to use direct SQL query Phase 8: Remove call to findActiveSDKSession() and replace with direct database query in SessionCompletionHandler.completeByClaudeId(). This removes dependency on the deleted findActiveSDKSession() method and simplifies the code by using a straightforward SELECT query. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: remove markSessionCompleted call from SDKAgent - Delete call to markSessionCompleted() in SDKAgent.ts - Session status is no longer tracked or updated - Part of phase 9: simplifying session management 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: remove markSessionComplete method (Phase 10) - Deleted markSessionComplete() method from DatabaseManager - Removed markSessionComplete call from SessionCompletionHandler - Session completion status no longer tracked in database - Part of session management simplification effort 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: replace deleted updateSDKSessionId calls in import script (Phase 11) - Replace updateSDKSessionId() calls with direct SQL UPDATE statements - Method was deleted in Phase 3 as part of session management simplification - Import script now uses direct database access consistently 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * test: add validation for SQL updates in sdk_sessions table * refactor: enhance worker-cli to support manual and automated runs * Remove cleanup hook and associated session completion logic - Deleted the cleanup-hook implementation from the hooks directory. - Removed the session completion endpoint that was used by the cleanup hook. - Updated the SessionCompletionHandler to eliminate the completeByClaudeId method and its dependencies. - Adjusted the SessionRoutes to reflect the removal of the session completion route. * fix: update worker-cli command to use bun for consistency * feat: Implement timestamp fix for observations and enhance processing logic - Added `earliestPendingTimestamp` to `ActiveSession` to track the original timestamp of the earliest pending message. - Updated `SDKAgent` to capture and utilize the earliest pending timestamp during response processing. - Modified `SessionManager` to track the earliest timestamp when yielding messages. - Created scripts for fixing corrupted timestamps, validating fixes, and investigating timestamp issues. - Verified that all corrupted observations have been repaired and logic for future processing is sound. - Ensured orphan processing can be safely re-enabled after validation. * feat: Enhance SessionStore to support custom database paths and add timestamp fields for observations and summaries * Refactor pending queue processing and add management endpoints - Disabled automatic recovery of orphaned queues on startup; users must now use the new /api/pending-queue/process endpoint. - Updated processOrphanedQueues method to processPendingQueues with improved session handling and return detailed results. - Added new API endpoints for managing pending queues: GET /api/pending-queue and POST /api/pending-queue/process. - Introduced a new script (check-pending-queue.ts) for checking and processing pending observation queues interactively or automatically. - Enhanced logging and error handling for better monitoring of session processing. * updated agent sdk * feat: Add manual recovery guide and queue management endpoints to documentation --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-25 15:36:46 -05:00
parent 4cf2c1bdb1
commit 266c746d50
47 changed files with 3417 additions and 1026 deletions
@@ -17,7 +17,6 @@ const HOOKS = [
  { name: 'new-hook', source: 'src/hooks/new-hook.ts' },
  { name: 'save-hook', source: 'src/hooks/save-hook.ts' },
  { name: 'summary-hook', source: 'src/hooks/summary-hook.ts' },
-  { name: 'cleanup-hook', source: 'src/hooks/cleanup-hook.ts' },
  { name: 'user-message-hook', source: 'src/hooks/user-message-hook.ts' }
 ];

@@ -0,0 +1,241 @@
+#!/usr/bin/env bun
+/**
+ * Check and process pending observation queue
+ *
+ * Usage:
+ *   bun scripts/check-pending-queue.ts           # Check status and prompt to process
+ *   bun scripts/check-pending-queue.ts --process # Auto-process without prompting
+ *   bun scripts/check-pending-queue.ts --limit 5 # Process up to 5 sessions
+ */
+
+const WORKER_URL = 'http://localhost:37777';
+
+interface QueueMessage {
+  id: number;
+  session_db_id: number;
+  message_type: string;
+  tool_name: string | null;
+  status: 'pending' | 'processing' | 'failed';
+  retry_count: number;
+  created_at_epoch: number;
+  project: string | null;
+}
+
+interface QueueResponse {
+  queue: {
+    messages: QueueMessage[];
+    totalPending: number;
+    totalProcessing: number;
+    totalFailed: number;
+    stuckCount: number;
+  };
+  recentlyProcessed: QueueMessage[];
+  sessionsWithPendingWork: number[];
+}
+
+interface ProcessResponse {
+  success: boolean;
+  totalPendingSessions: number;
+  sessionsStarted: number;
+  sessionsSkipped: number;
+  startedSessionIds: number[];
+}
+
+async function checkWorkerHealth(): Promise<boolean> {
+  try {
+    const res = await fetch(`${WORKER_URL}/api/health`);
+    return res.ok;
+  } catch {
+    return false;
+  }
+}
+
+async function getQueueStatus(): Promise<QueueResponse> {
+  const res = await fetch(`${WORKER_URL}/api/pending-queue`);
+  if (!res.ok) {
+    throw new Error(`Failed to get queue status: ${res.status}`);
+  }
+  return res.json();
+}
+
+async function processQueue(limit: number): Promise<ProcessResponse> {
+  const res = await fetch(`${WORKER_URL}/api/pending-queue/process`, {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({ sessionLimit: limit })
+  });
+  if (!res.ok) {
+    throw new Error(`Failed to process queue: ${res.status}`);
+  }
+  return res.json();
+}
+
+function formatAge(epochMs: number): string {
+  const ageMs = Date.now() - epochMs;
+  const minutes = Math.floor(ageMs / 60000);
+  const hours = Math.floor(minutes / 60);
+  const days = Math.floor(hours / 24);
+
+  if (days > 0) return `${days}d ${hours % 24}h ago`;
+  if (hours > 0) return `${hours}h ${minutes % 60}m ago`;
+  return `${minutes}m ago`;
+}
+
+async function prompt(question: string): Promise<string> {
+  // Check if we have a TTY for interactive input
+  if (!process.stdin.isTTY) {
+    console.log(question + '(no TTY, use --process flag for non-interactive mode)');
+    return 'n';
+  }
+
+  return new Promise((resolve) => {
+    process.stdout.write(question);
+    process.stdin.setRawMode(false);
+    process.stdin.resume();
+    process.stdin.once('data', (data) => {
+      process.stdin.pause();
+      resolve(data.toString().trim());
+    });
+  });
+}
+
+async function main() {
+  const args = process.argv.slice(2);
+
+  // Help flag
+  if (args.includes('--help') || args.includes('-h')) {
+    console.log(`
+Claude-Mem Pending Queue Manager
+
+Check and process pending observation queue backlog.
+
+Usage:
+  bun scripts/check-pending-queue.ts [options]
+
+Options:
+  --help, -h     Show this help message
+  --process      Auto-process without prompting
+  --limit N      Process up to N sessions (default: 10)
+
+Examples:
+  # Check queue status interactively
+  bun scripts/check-pending-queue.ts
+
+  # Auto-process up to 10 sessions
+  bun scripts/check-pending-queue.ts --process
+
+  # Process up to 5 sessions
+  bun scripts/check-pending-queue.ts --process --limit 5
+
+What is this for?
+  If the claude-mem worker crashes or restarts, pending observations may
+  be left unprocessed. This script shows the backlog and lets you trigger
+  processing. The worker no longer auto-recovers on startup to give you
+  control over when processing happens.
+`);
+    process.exit(0);
+  }
+
+  const autoProcess = args.includes('--process');
+  const limitArg = args.find((_, i) => args[i - 1] === '--limit');
+  const limit = limitArg ? parseInt(limitArg, 10) : 10;
+
+  console.log('\n=== Claude-Mem Pending Queue Status ===\n');
+
+  // Check worker health
+  const healthy = await checkWorkerHealth();
+  if (!healthy) {
+    console.log('Worker is not running. Start it with:');
+    console.log('  cd ~/.claude/plugins/marketplaces/thedotmack && npm run worker:start\n');
+    process.exit(1);
+  }
+  console.log('Worker status: Running\n');
+
+  // Get queue status
+  const status = await getQueueStatus();
+  const { queue, sessionsWithPendingWork } = status;
+
+  // Display summary
+  console.log('Queue Summary:');
+  console.log(`  Pending:    ${queue.totalPending}`);
+  console.log(`  Processing: ${queue.totalProcessing}`);
+  console.log(`  Failed:     ${queue.totalFailed}`);
+  console.log(`  Stuck:      ${queue.stuckCount} (processing > 5 min)`);
+  console.log(`  Sessions:   ${sessionsWithPendingWork.length} with pending work\n`);
+
+  // Check if there's any backlog
+  const hasBacklog = queue.totalPending > 0 || queue.totalFailed > 0;
+  const hasStuck = queue.stuckCount > 0;
+
+  if (!hasBacklog && !hasStuck) {
+    console.log('No backlog detected. Queue is healthy.\n');
+
+    // Show recently processed if any
+    if (status.recentlyProcessed.length > 0) {
+      console.log(`Recently processed: ${status.recentlyProcessed.length} messages in last 30 min\n`);
+    }
+    process.exit(0);
+  }
+
+  // Show details about pending messages
+  if (queue.messages.length > 0) {
+    console.log('Pending Messages:');
+    console.log('─'.repeat(80));
+
+    // Group by session
+    const bySession = new Map<number, QueueMessage[]>();
+    for (const msg of queue.messages) {
+      const list = bySession.get(msg.session_db_id) || [];
+      list.push(msg);
+      bySession.set(msg.session_db_id, list);
+    }
+
+    for (const [sessionId, messages] of bySession) {
+      const project = messages[0].project || 'unknown';
+      const oldest = Math.min(...messages.map(m => m.created_at_epoch));
+      const statuses = {
+        pending: messages.filter(m => m.status === 'pending').length,
+        processing: messages.filter(m => m.status === 'processing').length,
+        failed: messages.filter(m => m.status === 'failed').length
+      };
+
+      console.log(`  Session ${sessionId} (${project})`);
+      console.log(`    Messages: ${messages.length} total`);
+      console.log(`    Status:   ${statuses.pending} pending, ${statuses.processing} processing, ${statuses.failed} failed`);
+      console.log(`    Age:      ${formatAge(oldest)}`);
+    }
+    console.log('─'.repeat(80));
+    console.log('');
+  }
+
+  // Offer to process
+  if (autoProcess) {
+    console.log(`Auto-processing up to ${limit} sessions...\n`);
+  } else {
+    const answer = await prompt(`Process pending queue? (up to ${limit} sessions) [y/N]: `);
+    if (answer.toLowerCase() !== 'y') {
+      console.log('\nSkipped. Run with --process to auto-process.\n');
+      process.exit(0);
+    }
+    console.log('');
+  }
+
+  // Process the queue
+  const result = await processQueue(limit);
+
+  console.log('Processing Result:');
+  console.log(`  Sessions started: ${result.sessionsStarted}`);
+  console.log(`  Sessions skipped: ${result.sessionsSkipped} (already active)`);
+  console.log(`  Remaining:        ${result.totalPendingSessions - result.sessionsStarted}`);
+
+  if (result.startedSessionIds.length > 0) {
+    console.log(`  Started IDs:      ${result.startedSessionIds.join(', ')}`);
+  }
+
+  console.log('\nProcessing started in background. Check status again in a few minutes.\n');
+}
+
+main().catch(err => {
+  console.error('Error:', err.message);
+  process.exit(1);
+});
@@ -0,0 +1,174 @@
+#!/usr/bin/env bun
+
+/**
+ * Fix ALL Corrupted Observation Timestamps
+ *
+ * This script finds and repairs ALL observations with timestamps that don't match
+ * their session start times, not just ones in an arbitrary "bad window".
+ */
+
+import Database from 'bun:sqlite';
+import { resolve } from 'path';
+
+const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
+
+interface CorruptedObservation {
+  obs_id: number;
+  obs_title: string;
+  obs_created: number;
+  session_started: number;
+  session_completed: number | null;
+  sdk_session_id: string;
+}
+
+function formatTimestamp(epoch: number): string {
+  return new Date(epoch).toLocaleString('en-US', {
+    timeZone: 'America/Los_Angeles',
+    year: 'numeric',
+    month: 'short',
+    day: 'numeric',
+    hour: '2-digit',
+    minute: '2-digit',
+    second: '2-digit'
+  });
+}
+
+function main() {
+  const args = process.argv.slice(2);
+  const dryRun = args.includes('--dry-run');
+  const autoYes = args.includes('--yes') || args.includes('-y');
+
+  console.log('🔍 Finding ALL observations with timestamp corruption...\n');
+  if (dryRun) {
+    console.log('🏃 DRY RUN MODE - No changes will be made\n');
+  }
+
+  const db = new Database(DB_PATH);
+
+  try {
+    // Find all observations where timestamp doesn't match session
+    const corrupted = db.query<CorruptedObservation, []>(`
+      SELECT
+        o.id as obs_id,
+        o.title as obs_title,
+        o.created_at_epoch as obs_created,
+        s.started_at_epoch as session_started,
+        s.completed_at_epoch as session_completed,
+        s.sdk_session_id
+      FROM observations o
+      JOIN sdk_sessions s ON o.sdk_session_id = s.sdk_session_id
+      WHERE o.created_at_epoch < s.started_at_epoch  -- Observation older than session
+         OR (s.completed_at_epoch IS NOT NULL
+             AND o.created_at_epoch > (s.completed_at_epoch + 3600000))  -- More than 1hr after session
+      ORDER BY o.id
+    `).all();
+
+    console.log(`Found ${corrupted.length} observations with corrupted timestamps\n`);
+
+    if (corrupted.length === 0) {
+      console.log('✅ No corrupted timestamps found!');
+      db.close();
+      return;
+    }
+
+    // Display findings
+    console.log('═══════════════════════════════════════════════════════════════════════');
+    console.log('PROPOSED FIXES:');
+    console.log('═══════════════════════════════════════════════════════════════════════\n');
+
+    for (const obs of corrupted.slice(0, 50)) {
+      const daysDiff = Math.round((obs.obs_created - obs.session_started) / (1000 * 60 * 60 * 24));
+      console.log(`Observation #${obs.obs_id}: ${obs.obs_title || '(no title)'}`);
+      console.log(`  ❌ Wrong: ${formatTimestamp(obs.obs_created)}`);
+      console.log(`  ✅ Correct: ${formatTimestamp(obs.session_started)}`);
+      console.log(`  📅 Off by ${daysDiff} days\n`);
+    }
+
+    if (corrupted.length > 50) {
+      console.log(`... and ${corrupted.length - 50} more\n`);
+    }
+
+    console.log('═══════════════════════════════════════════════════════════════════════');
+    console.log(`Ready to fix ${corrupted.length} observations.`);
+
+    if (dryRun) {
+      console.log('\n🏃 DRY RUN COMPLETE - No changes made.');
+      console.log('Run without --dry-run flag to apply fixes.\n');
+      db.close();
+      return;
+    }
+
+    if (autoYes) {
+      console.log('Auto-confirming with --yes flag...\n');
+      applyFixes(db, corrupted);
+      return;
+    }
+
+    console.log('Apply these fixes? (y/n): ');
+
+    const stdin = Bun.stdin.stream();
+    const reader = stdin.getReader();
+
+    reader.read().then(({ value }) => {
+      const response = new TextDecoder().decode(value).trim().toLowerCase();
+
+      if (response === 'y' || response === 'yes') {
+        applyFixes(db, corrupted);
+      } else {
+        console.log('\n❌ Fixes cancelled. No changes made.');
+        db.close();
+      }
+    });
+
+  } catch (error) {
+    console.error('❌ Error:', error);
+    db.close();
+    process.exit(1);
+  }
+}
+
+function applyFixes(db: Database, corrupted: CorruptedObservation[]) {
+  console.log('\n🔧 Applying fixes...\n');
+
+  const updateStmt = db.prepare(`
+    UPDATE observations
+    SET created_at_epoch = ?,
+        created_at = datetime(?/1000, 'unixepoch')
+    WHERE id = ?
+  `);
+
+  let successCount = 0;
+  let errorCount = 0;
+
+  for (const obs of corrupted) {
+    try {
+      updateStmt.run(
+        obs.session_started,
+        obs.session_started,
+        obs.obs_id
+      );
+      successCount++;
+      if (successCount % 10 === 0 || successCount <= 10) {
+        console.log(`✅ Fixed observation #${obs.obs_id}`);
+      }
+    } catch (error) {
+      errorCount++;
+      console.error(`❌ Failed to fix observation #${obs.obs_id}:`, error);
+    }
+  }
+
+  console.log('\n═══════════════════════════════════════════════════════════════════════');
+  console.log('RESULTS:');
+  console.log('═══════════════════════════════════════════════════════════════════════');
+  console.log(`✅ Successfully fixed: ${successCount}`);
+  console.log(`❌ Failed: ${errorCount}`);
+  console.log(`📊 Total processed: ${corrupted.length}\n`);
+
+  if (successCount > 0) {
+    console.log('🎉 ALL timestamp corruption has been repaired!\n');
+  }
+
+  db.close();
+}
+
+main();
@@ -0,0 +1,243 @@
+#!/usr/bin/env bun
+
+/**
+ * Fix Corrupted Observation Timestamps
+ *
+ * This script repairs observations that were created during the orphan queue processing
+ * on Dec 24, 2025 between 19:45-20:31. These observations got Dec 24 timestamps instead
+ * of their original timestamps from Dec 17-20.
+ */
+
+import Database from 'bun:sqlite';
+import { resolve } from 'path';
+
+const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
+
+// Bad window: Dec 24 19:45-20:31 (timestamps in milliseconds, not microseconds)
+// Using actual observation epoch format (microseconds since epoch)
+const BAD_WINDOW_START = 1766623500000; // Dec 24 19:45 PST
+const BAD_WINDOW_END = 1766626260000;   // Dec 24 20:31 PST
+
+interface AffectedObservation {
+  id: number;
+  sdk_session_id: string;
+  created_at_epoch: number;
+  title: string;
+}
+
+interface ProcessedMessage {
+  id: number;
+  session_db_id: number;
+  tool_name: string;
+  created_at_epoch: number;
+  completed_at_epoch: number;
+}
+
+interface SessionMapping {
+  session_db_id: number;
+  sdk_session_id: string;
+}
+
+interface TimestampFix {
+  observation_id: number;
+  observation_title: string;
+  wrong_timestamp: number;
+  correct_timestamp: number;
+  session_db_id: number;
+  pending_message_id: number;
+}
+
+function formatTimestamp(epoch: number): string {
+  return new Date(epoch).toLocaleString('en-US', {
+    timeZone: 'America/Los_Angeles',
+    year: 'numeric',
+    month: 'short',
+    day: 'numeric',
+    hour: '2-digit',
+    minute: '2-digit',
+    second: '2-digit'
+  });
+}
+
+function main() {
+  const args = process.argv.slice(2);
+  const dryRun = args.includes('--dry-run');
+  const autoYes = args.includes('--yes') || args.includes('-y');
+
+  console.log('🔍 Analyzing corrupted observation timestamps...\n');
+  if (dryRun) {
+    console.log('🏃 DRY RUN MODE - No changes will be made\n');
+  }
+
+  const db = new Database(DB_PATH);
+
+  try {
+    // Step 1: Find affected observations
+    console.log('Step 1: Finding observations created during bad window...');
+    const affectedObs = db.query<AffectedObservation, []>(`
+      SELECT id, sdk_session_id, created_at_epoch, title
+      FROM observations
+      WHERE created_at_epoch >= ${BAD_WINDOW_START}
+        AND created_at_epoch <= ${BAD_WINDOW_END}
+      ORDER BY id
+    `).all();
+
+    console.log(`Found ${affectedObs.length} observations in bad window\n`);
+
+    if (affectedObs.length === 0) {
+      console.log('✅ No affected observations found!');
+      return;
+    }
+
+    // Step 2: Find processed pending_messages from bad window
+    console.log('Step 2: Finding pending messages processed during bad window...');
+    const processedMessages = db.query<ProcessedMessage, []>(`
+      SELECT id, session_db_id, tool_name, created_at_epoch, completed_at_epoch
+      FROM pending_messages
+      WHERE status = 'processed'
+        AND completed_at_epoch >= ${BAD_WINDOW_START}
+        AND completed_at_epoch <= ${BAD_WINDOW_END}
+      ORDER BY completed_at_epoch
+    `).all();
+
+    console.log(`Found ${processedMessages.length} processed messages\n`);
+
+    // Step 3: Match observations to their session start times (simpler approach)
+    console.log('Step 3: Matching observations to session start times...');
+    const fixes: TimestampFix[] = [];
+
+    interface ObsWithSession {
+      obs_id: number;
+      obs_title: string;
+      obs_created: number;
+      session_started: number;
+      sdk_session_id: string;
+    }
+
+    const obsWithSessions = db.query<ObsWithSession, []>(`
+      SELECT
+        o.id as obs_id,
+        o.title as obs_title,
+        o.created_at_epoch as obs_created,
+        s.started_at_epoch as session_started,
+        s.sdk_session_id
+      FROM observations o
+      JOIN sdk_sessions s ON o.sdk_session_id = s.sdk_session_id
+      WHERE o.created_at_epoch >= ${BAD_WINDOW_START}
+        AND o.created_at_epoch <= ${BAD_WINDOW_END}
+        AND s.started_at_epoch < ${BAD_WINDOW_START}
+      ORDER BY o.id
+    `).all();
+
+    for (const row of obsWithSessions) {
+      fixes.push({
+        observation_id: row.obs_id,
+        observation_title: row.obs_title || '(no title)',
+        wrong_timestamp: row.obs_created,
+        correct_timestamp: row.session_started,
+        session_db_id: 0, // Not needed for this approach
+        pending_message_id: 0 // Not needed for this approach
+      });
+    }
+
+    console.log(`Identified ${fixes.length} observations to fix\n`);
+
+    // Step 5: Display what will be fixed
+    console.log('═══════════════════════════════════════════════════════════════════════');
+    console.log('PROPOSED FIXES:');
+    console.log('═══════════════════════════════════════════════════════════════════════\n');
+
+    for (const fix of fixes) {
+      const daysDiff = Math.round((fix.wrong_timestamp - fix.correct_timestamp) / (1000 * 60 * 60 * 24));
+      console.log(`Observation #${fix.observation_id}: ${fix.observation_title}`);
+      console.log(`  ❌ Wrong: ${formatTimestamp(fix.wrong_timestamp)}`);
+      console.log(`  ✅ Correct: ${formatTimestamp(fix.correct_timestamp)}`);
+      console.log(`  📅 Off by ${daysDiff} days\n`);
+    }
+
+    // Step 6: Ask for confirmation
+    console.log('═══════════════════════════════════════════════════════════════════════');
+    console.log(`Ready to fix ${fixes.length} observations.`);
+
+    if (dryRun) {
+      console.log('\n🏃 DRY RUN COMPLETE - No changes made.');
+      console.log('Run without --dry-run flag to apply fixes.\n');
+      db.close();
+      return;
+    }
+
+    if (autoYes) {
+      console.log('Auto-confirming with --yes flag...\n');
+      applyFixes(db, fixes);
+      return;
+    }
+
+    console.log('Apply these fixes? (y/n): ');
+
+    const stdin = Bun.stdin.stream();
+    const reader = stdin.getReader();
+
+    reader.read().then(({ value }) => {
+      const response = new TextDecoder().decode(value).trim().toLowerCase();
+
+      if (response === 'y' || response === 'yes') {
+        applyFixes(db, fixes);
+      } else {
+        console.log('\n❌ Fixes cancelled. No changes made.');
+        db.close();
+      }
+    });
+
+  } catch (error) {
+    console.error('❌ Error:', error);
+    db.close();
+    process.exit(1);
+  }
+}
+
+function applyFixes(db: Database, fixes: TimestampFix[]) {
+  console.log('\n🔧 Applying fixes...\n');
+
+  const updateStmt = db.prepare(`
+    UPDATE observations
+    SET created_at_epoch = ?,
+        created_at = datetime(?/1000, 'unixepoch')
+    WHERE id = ?
+  `);
+
+  let successCount = 0;
+  let errorCount = 0;
+
+  for (const fix of fixes) {
+    try {
+      updateStmt.run(
+        fix.correct_timestamp,
+        fix.correct_timestamp,
+        fix.observation_id
+      );
+      successCount++;
+      console.log(`✅ Fixed observation #${fix.observation_id}`);
+    } catch (error) {
+      errorCount++;
+      console.error(`❌ Failed to fix observation #${fix.observation_id}:`, error);
+    }
+  }
+
+  console.log('\n═══════════════════════════════════════════════════════════════════════');
+  console.log('RESULTS:');
+  console.log('═══════════════════════════════════════════════════════════════════════');
+  console.log(`✅ Successfully fixed: ${successCount}`);
+  console.log(`❌ Failed: ${errorCount}`);
+  console.log(`📊 Total processed: ${fixes.length}\n`);
+
+  if (successCount > 0) {
+    console.log('🎉 Timestamp corruption has been repaired!');
+    console.log('💡 Next steps:');
+    console.log('   1. Verify the fixes with: bun scripts/verify-timestamp-fix.ts');
+    console.log('   2. Consider re-enabling orphan processing if timestamp fix is working\n');
+  }
+
+  db.close();
+}
+
+main();
@@ -0,0 +1,143 @@
+#!/usr/bin/env bun
+
+/**
+ * Investigate Timestamp Situation
+ *
+ * This script investigates the actual state of observations and pending messages
+ * to understand what happened with the timestamp corruption.
+ */
+
+import Database from 'bun:sqlite';
+import { resolve } from 'path';
+
+const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
+
+function formatTimestamp(epoch: number): string {
+  return new Date(epoch).toLocaleString('en-US', {
+    timeZone: 'America/Los_Angeles',
+    year: 'numeric',
+    month: 'short',
+    day: 'numeric',
+    hour: '2-digit',
+    minute: '2-digit',
+    second: '2-digit'
+  });
+}
+
+function main() {
+  console.log('🔍 Investigating timestamp situation...\n');
+
+  const db = new Database(DB_PATH);
+
+  try {
+    // Check 1: Recent observations on Dec 24
+    console.log('Check 1: All observations created on Dec 24, 2025...');
+    const dec24Start = 1735027200000; // Dec 24 00:00 PST
+    const dec24End = 1735113600000;   // Dec 25 00:00 PST
+
+    const dec24Obs = db.query(`
+      SELECT id, sdk_session_id, created_at_epoch, title
+      FROM observations
+      WHERE created_at_epoch >= ${dec24Start}
+        AND created_at_epoch < ${dec24End}
+      ORDER BY created_at_epoch
+      LIMIT 100
+    `).all();
+
+    console.log(`Found ${dec24Obs.length} observations on Dec 24:\n`);
+    for (const obs of dec24Obs.slice(0, 20)) {
+      console.log(`  #${obs.id}: ${formatTimestamp(obs.created_at_epoch)} - ${obs.title || '(no title)'}`);
+    }
+    if (dec24Obs.length > 20) {
+      console.log(`  ... and ${dec24Obs.length - 20} more`);
+    }
+    console.log();
+
+    // Check 2: Observations from Dec 17-20
+    console.log('Check 2: Observations from Dec 17-20, 2025...');
+    const dec17Start = 1734422400000; // Dec 17 00:00 PST
+    const dec21Start = 1734768000000; // Dec 21 00:00 PST
+
+    const oldObs = db.query(`
+      SELECT id, sdk_session_id, created_at_epoch, title
+      FROM observations
+      WHERE created_at_epoch >= ${dec17Start}
+        AND created_at_epoch < ${dec21Start}
+      ORDER BY created_at_epoch
+      LIMIT 100
+    `).all();
+
+    console.log(`Found ${oldObs.length} observations from Dec 17-20:\n`);
+    for (const obs of oldObs.slice(0, 20)) {
+      console.log(`  #${obs.id}: ${formatTimestamp(obs.created_at_epoch)} - ${obs.title || '(no title)'}`);
+    }
+    if (oldObs.length > 20) {
+      console.log(`  ... and ${oldObs.length - 20} more`);
+    }
+    console.log();
+
+    // Check 3: Pending messages status
+    console.log('Check 3: Pending messages status...');
+    const statusCounts = db.query(`
+      SELECT status, COUNT(*) as count
+      FROM pending_messages
+      GROUP BY status
+    `).all();
+
+    console.log('Pending message counts by status:');
+    for (const row of statusCounts) {
+      console.log(`  ${row.status}: ${row.count}`);
+    }
+    console.log();
+
+    // Check 4: Old pending messages from Dec 17-20
+    console.log('Check 4: Pending messages from Dec 17-20...');
+    const oldMessages = db.query(`
+      SELECT id, session_db_id, tool_name, status, created_at_epoch, completed_at_epoch
+      FROM pending_messages
+      WHERE created_at_epoch >= ${dec17Start}
+        AND created_at_epoch < ${dec21Start}
+      ORDER BY created_at_epoch
+      LIMIT 50
+    `).all();
+
+    console.log(`Found ${oldMessages.length} pending messages from Dec 17-20:\n`);
+    for (const msg of oldMessages.slice(0, 20)) {
+      const completedAt = msg.completed_at_epoch ? formatTimestamp(msg.completed_at_epoch) : 'N/A';
+      console.log(`  #${msg.id}: ${msg.tool_name} - Status: ${msg.status}`);
+      console.log(`    Created: ${formatTimestamp(msg.created_at_epoch)}`);
+      console.log(`    Completed: ${completedAt}\n`);
+    }
+    if (oldMessages.length > 20) {
+      console.log(`  ... and ${oldMessages.length - 20} more`);
+    }
+
+    // Check 5: Recently completed pending messages
+    console.log('Check 5: Recently completed pending messages...');
+    const recentCompleted = db.query(`
+      SELECT id, session_db_id, tool_name, status, created_at_epoch, completed_at_epoch
+      FROM pending_messages
+      WHERE completed_at_epoch IS NOT NULL
+      ORDER BY completed_at_epoch DESC
+      LIMIT 20
+    `).all();
+
+    console.log(`Most recent completed pending messages:\n`);
+    for (const msg of recentCompleted) {
+      const createdAt = formatTimestamp(msg.created_at_epoch);
+      const completedAt = formatTimestamp(msg.completed_at_epoch);
+      const lag = Math.round((msg.completed_at_epoch - msg.created_at_epoch) / 1000);
+      console.log(`  #${msg.id}: ${msg.tool_name} (${msg.status})`);
+      console.log(`    Created: ${createdAt}`);
+      console.log(`    Completed: ${completedAt} (${lag}s later)\n`);
+    }
+
+  } catch (error) {
+    console.error('❌ Error:', error);
+    process.exit(1);
+  } finally {
+    db.close();
+  }
+}
+
+main();
@@ -0,0 +1,150 @@
+#!/usr/bin/env bun
+
+/**
+ * Validate Timestamp Logic
+ *
+ * This script validates that the backlog timestamp logic would work correctly
+ * by checking pending messages and simulating what timestamps they would get.
+ */
+
+import Database from 'bun:sqlite';
+import { resolve } from 'path';
+
+const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
+
+function formatTimestamp(epoch: number): string {
+  return new Date(epoch).toLocaleString('en-US', {
+    timeZone: 'America/Los_Angeles',
+    year: 'numeric',
+    month: 'short',
+    day: 'numeric',
+    hour: '2-digit',
+    minute: '2-digit',
+    second: '2-digit'
+  });
+}
+
+function main() {
+  console.log('🔍 Validating timestamp logic for backlog processing...\n');
+
+  const db = new Database(DB_PATH);
+
+  try {
+    // Check for pending messages
+    const pendingStats = db.query(`
+      SELECT
+        status,
+        COUNT(*) as count,
+        MIN(created_at_epoch) as earliest,
+        MAX(created_at_epoch) as latest
+      FROM pending_messages
+      GROUP BY status
+      ORDER BY status
+    `).all();
+
+    console.log('Pending Messages Status:\n');
+    for (const stat of pendingStats) {
+      console.log(`${stat.status}: ${stat.count} messages`);
+      if (stat.earliest && stat.latest) {
+        console.log(`  Created: ${formatTimestamp(stat.earliest)} to ${formatTimestamp(stat.latest)}`);
+      }
+    }
+    console.log();
+
+    // Get sample pending messages with their session info
+    const pendingWithSessions = db.query(`
+      SELECT
+        pm.id,
+        pm.session_db_id,
+        pm.tool_name,
+        pm.created_at_epoch as msg_created,
+        pm.status,
+        s.sdk_session_id,
+        s.started_at_epoch as session_started,
+        s.project
+      FROM pending_messages pm
+      LEFT JOIN sdk_sessions s ON pm.session_db_id = s.id
+      WHERE pm.status IN ('pending', 'processing')
+      ORDER BY pm.created_at_epoch
+      LIMIT 10
+    `).all();
+
+    if (pendingWithSessions.length === 0) {
+      console.log('✅ No pending messages - all caught up!\n');
+      db.close();
+      return;
+    }
+
+    console.log(`Sample of ${pendingWithSessions.length} pending messages:\n`);
+    console.log('═══════════════════════════════════════════════════════════════════════');
+
+    for (const msg of pendingWithSessions) {
+      console.log(`\nPending Message #${msg.id}: ${msg.tool_name} (${msg.status})`);
+      console.log(`  Created: ${formatTimestamp(msg.msg_created)}`);
+
+      if (msg.session_started) {
+        console.log(`  Session started: ${formatTimestamp(msg.session_started)}`);
+        console.log(`  Project: ${msg.project}`);
+
+        // Validate logic
+        const ageDays = Math.round((Date.now() - msg.msg_created) / (1000 * 60 * 60 * 24));
+
+        if (msg.msg_created < msg.session_started) {
+          console.log(`  ⚠️  WARNING: Message created BEFORE session! This is impossible.`);
+        } else if (ageDays > 0) {
+          console.log(`  📅 Message is ${ageDays} days old`);
+          console.log(`  ✅ Would use original timestamp: ${formatTimestamp(msg.msg_created)}`);
+        } else {
+          console.log(`  ✅ Recent message, would use original timestamp: ${formatTimestamp(msg.msg_created)}`);
+        }
+      } else {
+        console.log(`  ⚠️  No session found for session_db_id ${msg.session_db_id}`);
+      }
+    }
+
+    console.log('\n═══════════════════════════════════════════════════════════════════════');
+    console.log('\nTimestamp Logic Validation:\n');
+    console.log('✅ Code Flow:');
+    console.log('   1. SessionManager.yieldNextMessage() tracks earliestPendingTimestamp');
+    console.log('   2. SDKAgent captures originalTimestamp before processing');
+    console.log('   3. processSDKResponse passes originalTimestamp to storeObservation/storeSummary');
+    console.log('   4. SessionStore uses overrideTimestampEpoch ?? Date.now()');
+    console.log('   5. earliestPendingTimestamp reset after batch completes\n');
+
+    console.log('✅ Expected Behavior:');
+    console.log('   - New messages: get current timestamp');
+    console.log('   - Backlog messages: get original created_at_epoch');
+    console.log('   - Observations match their source message timestamps\n');
+
+    // Check for any sessions with stuck processing messages
+    const stuckMessages = db.query(`
+      SELECT
+        session_db_id,
+        COUNT(*) as count,
+        MIN(created_at_epoch) as earliest,
+        MAX(created_at_epoch) as latest
+      FROM pending_messages
+      WHERE status = 'processing'
+      GROUP BY session_db_id
+      ORDER BY count DESC
+    `).all();
+
+    if (stuckMessages.length > 0) {
+      console.log('⚠️  Stuck Messages (status=processing):\n');
+      for (const stuck of stuckMessages) {
+        const ageDays = Math.round((Date.now() - stuck.earliest) / (1000 * 60 * 60 * 24));
+        console.log(`   Session ${stuck.session_db_id}: ${stuck.count} messages`);
+        console.log(`     Stuck for ${ageDays} days (${formatTimestamp(stuck.earliest)})`);
+      }
+      console.log('\n   💡 These will be processed with original timestamps when orphan processing is enabled\n');
+    }
+
+  } catch (error) {
+    console.error('❌ Error:', error);
+    process.exit(1);
+  } finally {
+    db.close();
+  }
+}
+
+main();
@@ -0,0 +1,144 @@
+#!/usr/bin/env bun
+
+/**
+ * Verify Timestamp Fix
+ *
+ * This script verifies that the timestamp corruption has been properly fixed.
+ * It checks for any remaining observations in the bad window that shouldn't be there.
+ */
+
+import Database from 'bun:sqlite';
+import { resolve } from 'path';
+
+const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
+
+// Bad window: Dec 24 19:45-20:31 (using actual epoch format from database)
+const BAD_WINDOW_START = 1766623500000; // Dec 24 19:45 PST
+const BAD_WINDOW_END = 1766626260000;   // Dec 24 20:31 PST
+
+// Original corruption window: Dec 16-22 (when sessions actually started)
+const ORIGINAL_WINDOW_START = 1765914000000; // Dec 16 00:00 PST
+const ORIGINAL_WINDOW_END = 1766613600000;   // Dec 23 23:59 PST
+
+interface Observation {
+  id: number;
+  sdk_session_id: string;
+  created_at_epoch: number;
+  created_at: string;
+  title: string;
+}
+
+function formatTimestamp(epoch: number): string {
+  return new Date(epoch).toLocaleString('en-US', {
+    timeZone: 'America/Los_Angeles',
+    year: 'numeric',
+    month: 'short',
+    day: 'numeric',
+    hour: '2-digit',
+    minute: '2-digit',
+    second: '2-digit'
+  });
+}
+
+function main() {
+  console.log('🔍 Verifying timestamp fix...\n');
+
+  const db = new Database(DB_PATH);
+
+  try {
+    // Check 1: Observations still in bad window
+    console.log('Check 1: Looking for observations still in bad window (Dec 24 19:45-20:31)...');
+    const badWindowObs = db.query<Observation, []>(`
+      SELECT id, sdk_session_id, created_at_epoch, created_at, title
+      FROM observations
+      WHERE created_at_epoch >= ${BAD_WINDOW_START}
+        AND created_at_epoch <= ${BAD_WINDOW_END}
+      ORDER BY id
+    `).all();
+
+    if (badWindowObs.length === 0) {
+      console.log('✅ No observations found in bad window - GOOD!\n');
+    } else {
+      console.log(`⚠️  Found ${badWindowObs.length} observations still in bad window:\n`);
+      for (const obs of badWindowObs) {
+        console.log(`  Observation #${obs.id}: ${obs.title || '(no title)'}`);
+        console.log(`    Timestamp: ${formatTimestamp(obs.created_at_epoch)}`);
+        console.log(`    Session: ${obs.sdk_session_id}\n`);
+      }
+    }
+
+    // Check 2: Observations now in original window
+    console.log('Check 2: Counting observations in original window (Dec 17-20)...');
+    const originalWindowObs = db.query<{ count: number }, []>(`
+      SELECT COUNT(*) as count
+      FROM observations
+      WHERE created_at_epoch >= ${ORIGINAL_WINDOW_START}
+        AND created_at_epoch <= ${ORIGINAL_WINDOW_END}
+    `).get();
+
+    console.log(`Found ${originalWindowObs?.count || 0} observations in Dec 17-20 window`);
+    console.log('(These should be the corrected observations)\n');
+
+    // Check 3: Session distribution
+    console.log('Check 3: Session distribution of corrected observations...');
+    const sessionDist = db.query<{ sdk_session_id: string; count: number }, []>(`
+      SELECT sdk_session_id, COUNT(*) as count
+      FROM observations
+      WHERE created_at_epoch >= ${ORIGINAL_WINDOW_START}
+        AND created_at_epoch <= ${ORIGINAL_WINDOW_END}
+      GROUP BY sdk_session_id
+      ORDER BY count DESC
+    `).all();
+
+    if (sessionDist.length > 0) {
+      console.log(`Observations distributed across ${sessionDist.length} sessions:\n`);
+      for (const dist of sessionDist.slice(0, 10)) {
+        console.log(`  ${dist.sdk_session_id}: ${dist.count} observations`);
+      }
+      if (sessionDist.length > 10) {
+        console.log(`  ... and ${sessionDist.length - 10} more sessions`);
+      }
+      console.log();
+    }
+
+    // Check 4: Pending messages processed count
+    console.log('Check 4: Verifying processed pending_messages...');
+    const processedCount = db.query<{ count: number }, []>(`
+      SELECT COUNT(*) as count
+      FROM pending_messages
+      WHERE status = 'processed'
+        AND completed_at_epoch >= ${BAD_WINDOW_START}
+        AND completed_at_epoch <= ${BAD_WINDOW_END}
+    `).get();
+
+    console.log(`${processedCount?.count || 0} pending messages were processed during bad window\n`);
+
+    // Summary
+    console.log('═══════════════════════════════════════════════════════════════════════');
+    console.log('VERIFICATION SUMMARY:');
+    console.log('═══════════════════════════════════════════════════════════════════════\n');
+
+    if (badWindowObs.length === 0 && (originalWindowObs?.count || 0) > 0) {
+      console.log('✅ SUCCESS: Timestamp fix appears to be working correctly!');
+      console.log(`   - No observations remain in bad window (Dec 24 19:45-20:31)`);
+      console.log(`   - ${originalWindowObs?.count} observations restored to Dec 17-20`);
+      console.log(`   - Processed ${processedCount?.count} pending messages`);
+      console.log('\n💡 Safe to re-enable orphan processing in worker-service.ts\n');
+    } else if (badWindowObs.length > 0) {
+      console.log('⚠️  WARNING: Some observations still have incorrect timestamps!');
+      console.log(`   - ${badWindowObs.length} observations still in bad window`);
+      console.log('   - Run fix-corrupted-timestamps.ts again or investigate manually\n');
+    } else {
+      console.log('ℹ️  No corrupted observations detected');
+      console.log('   - Either already fixed or corruption never occurred\n');
+    }
+
+  } catch (error) {
+    console.error('❌ Error:', error);
+    process.exit(1);
+  } finally {
+    db.close();
+  }
+}
+
+main();