claude-mem/TIMESTAMP-FIX-VALIDATION.md

Timestamp Fix Validation Report

Date: Dec 24, 2025 Status: ✅ VALIDATED - Logic is correct and working

Summary

The backlog timestamp fix has been validated. All 171 corrupted observations have been repaired, and the code logic for preventing future corruption is correct.

What Was Fixed

• Total corrupted observations: 171
• Date range: Oct 26 - Dec 24, 2025
• Root cause: Observations from old sessions being processed late got current timestamps instead of original timestamps
• Fix applied: Restored all observations to their correct original timestamps

Validation Results

1. Database Integrity ✅

Corrupted observations remaining: 0
Pending messages with issues: 0

2. Code Logic ✅

The timestamp override logic flows correctly:

1. SessionManager.yieldNextMessage() (src/services/worker/SessionManager.ts:451-454)

   • Tracks earliestPendingTimestamp when yielding messages
   • Uses Math.min() to keep the earliest timestamp across batches

2. SDKAgent.createMessageGenerator() (src/services/worker/SDKAgent.ts:209)

   • Calls getMessageIterator() which yields messages
   • earliestPendingTimestamp is set BEFORE messages are sent to Claude

3. SDKAgent response handling (src/services/worker/SDKAgent.ts:119)

   • Captures originalTimestamp = session.earliestPendingTimestamp
   • This happens when Claude RESPONDS, after messages were already yielded

4. SDKAgent.processSDKResponse() (src/services/worker/SDKAgent.ts:272, 350)

   • Passes originalTimestamp ?? undefined to storage methods
   • Both observations and summaries use this timestamp

5. SessionStore.storeObservation/storeSummary() (src/services/sqlite/SessionStore.ts:1157, 1210)

   • Uses overrideTimestampEpoch ?? Date.now()
   • If override provided (from backlog), uses that
   • Otherwise uses current time (for new messages)

6. SDKAgent.markMessagesProcessed() (src/services/worker/SDKAgent.ts:430)

   • Resets earliestPendingTimestamp = null after batch completes
   • Ready for next batch with fresh timestamp tracking

3. Sequence Validation ✅

Correct sequence for backlog messages:

Time    Event                                           earliestPendingTimestamp
------  ----------------------------------------------  ------------------------
T1      yieldNextMessage() called                       → Set to msg.created_at_epoch
T2      Messages sent to Claude SDK                     → Still set
T3      Claude responds                                 → Still set
T4      Capture originalTimestamp                       → Captured (equals T1 timestamp)
T5      Create observations with originalTimestamp      → Uses T1 timestamp ✅
T6      Mark messages processed                         → Reset to null

Correct sequence for new messages:

Time    Event                                           earliestPendingTimestamp
------  ----------------------------------------------  ------------------------
T1      yieldNextMessage() called (recent message)      → Set to msg.created_at_epoch (recent)
T2      Messages sent to Claude SDK                     → Still set
T3      Claude responds                                 → Still set
T4      Capture originalTimestamp                       → Captured (equals T1 timestamp)
T5      Create observations with originalTimestamp      → Uses T1 timestamp ✅
T6      Mark messages processed                         → Reset to null

In both cases, observations get the timestamp from when the message was originally created, not when the observation was saved.

Current State

Pending Messages

• 6 pending messages (all from Dec 24, 2025)
• 0 stuck messages (status='processing')
• All pending messages would be processed with correct timestamps if orphan processing enabled

Orphan Processing

• Currently DISABLED in src/services/worker-service.ts:479
• Safe to re-enable - timestamp fix is working correctly
• No risk of future timestamp corruption

Scripts Created

1. scripts/fix-all-timestamps.ts - Comprehensive fix for ALL corrupted timestamps

   bun scripts/fix-all-timestamps.ts --dry-run  # Preview
   bun scripts/fix-all-timestamps.ts --yes      # Apply
   

2. scripts/validate-timestamp-logic.ts - Validate the backlog timestamp logic

   bun scripts/validate-timestamp-logic.ts
   

3. scripts/verify-timestamp-fix.ts - Verify specific time window

   bun scripts/verify-timestamp-fix.ts
   

Recommendation

✅ Safe to re-enable orphan processing

The timestamp fix is working correctly. To re-enable:

// src/services/worker-service.ts:479
// Change from:
// this.processOrphanedQueues(pendingStore).catch((err: Error) => {

// To:
this.processOrphanedQueues(pendingStore).catch((err: Error) => {
  logger.warn('SYSTEM', 'Orphan queue processing failed', {}, err);
});

Files Changed in Fix

• src/services/sqlite/SessionStore.ts - Added overrideTimestampEpoch parameter
• src/services/worker-types.ts - Added earliestPendingTimestamp to ActiveSession
• src/services/worker/SessionManager.ts - Tracks earliest timestamp when yielding
• src/services/worker/SDKAgent.ts - Passes timestamp through to storage
• src/services/worker-service.ts - Orphan processing (currently disabled)

Conclusion

✅ All corrupted timestamps fixed (171 observations) ✅ Code logic validated and working correctly ✅ No remaining timestamp issues in database ✅ Safe to re-enable orphan processing