266c746d50
* Refactor worker version checks and increase timeout settings - Updated the default hook timeout from 5000ms to 120000ms for improved stability. - Modified the worker version check to log a warning instead of restarting the worker on version mismatch. - Removed legacy PM2 cleanup and worker start logic, simplifying the ensureWorkerRunning function. - Enhanced polling mechanism for worker readiness with increased retries and reduced interval. * feat: implement worker queue polling to ensure processing completion before proceeding * refactor: change worker command from start to restart in hooks configuration * refactor: remove session management complexity - Simplify createSDKSession to pure INSERT OR IGNORE - Remove auto-create logic from storeObservation/storeSummary - Delete 11 unused session management methods - Derive prompt_number from user_prompts count - Keep sdk_sessions table schema unchanged for compatibility * refactor: simplify session management by removing unused methods and auto-creation logic * Refactor session prompt number retrieval in SessionRoutes - Updated the method of obtaining the prompt number from the session. - Replaced `store.getPromptCounter(sessionDbId)` with `store.getPromptNumberFromUserPrompts(claudeSessionId)` for better clarity and accuracy. - Adjusted the logic for incrementing the prompt number to derive it from the user prompts count instead of directly incrementing a counter. * refactor: replace getPromptCounter with getPromptNumberFromUserPrompts in SessionManager Phase 7 of session management simplification. Updates SessionManager to derive prompt numbers from user_prompts table count instead of using the deprecated prompt_counter column. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: simplify SessionCompletionHandler to use direct SQL query Phase 8: Remove call to findActiveSDKSession() and replace with direct database query in SessionCompletionHandler.completeByClaudeId(). This removes dependency on the deleted findActiveSDKSession() method and simplifies the code by using a straightforward SELECT query. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: remove markSessionCompleted call from SDKAgent - Delete call to markSessionCompleted() in SDKAgent.ts - Session status is no longer tracked or updated - Part of phase 9: simplifying session management 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: remove markSessionComplete method (Phase 10) - Deleted markSessionComplete() method from DatabaseManager - Removed markSessionComplete call from SessionCompletionHandler - Session completion status no longer tracked in database - Part of session management simplification effort 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: replace deleted updateSDKSessionId calls in import script (Phase 11) - Replace updateSDKSessionId() calls with direct SQL UPDATE statements - Method was deleted in Phase 3 as part of session management simplification - Import script now uses direct database access consistently 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * test: add validation for SQL updates in sdk_sessions table * refactor: enhance worker-cli to support manual and automated runs * Remove cleanup hook and associated session completion logic - Deleted the cleanup-hook implementation from the hooks directory. - Removed the session completion endpoint that was used by the cleanup hook. - Updated the SessionCompletionHandler to eliminate the completeByClaudeId method and its dependencies. - Adjusted the SessionRoutes to reflect the removal of the session completion route. * fix: update worker-cli command to use bun for consistency * feat: Implement timestamp fix for observations and enhance processing logic - Added `earliestPendingTimestamp` to `ActiveSession` to track the original timestamp of the earliest pending message. - Updated `SDKAgent` to capture and utilize the earliest pending timestamp during response processing. - Modified `SessionManager` to track the earliest timestamp when yielding messages. - Created scripts for fixing corrupted timestamps, validating fixes, and investigating timestamp issues. - Verified that all corrupted observations have been repaired and logic for future processing is sound. - Ensured orphan processing can be safely re-enabled after validation. * feat: Enhance SessionStore to support custom database paths and add timestamp fields for observations and summaries * Refactor pending queue processing and add management endpoints - Disabled automatic recovery of orphaned queues on startup; users must now use the new /api/pending-queue/process endpoint. - Updated processOrphanedQueues method to processPendingQueues with improved session handling and return detailed results. - Added new API endpoints for managing pending queues: GET /api/pending-queue and POST /api/pending-queue/process. - Introduced a new script (check-pending-queue.ts) for checking and processing pending observation queues interactively or automatically. - Enhanced logging and error handling for better monitoring of session processing. * updated agent sdk * feat: Add manual recovery guide and queue management endpoints to documentation --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
151 lines
5.4 KiB
TypeScript
Executable File
151 lines
5.4 KiB
TypeScript
Executable File
#!/usr/bin/env bun
|
|
|
|
/**
|
|
* Validate Timestamp Logic
|
|
*
|
|
* This script validates that the backlog timestamp logic would work correctly
|
|
* by checking pending messages and simulating what timestamps they would get.
|
|
*/
|
|
|
|
import Database from 'bun:sqlite';
|
|
import { resolve } from 'path';
|
|
|
|
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
|
|
|
function formatTimestamp(epoch: number): string {
|
|
return new Date(epoch).toLocaleString('en-US', {
|
|
timeZone: 'America/Los_Angeles',
|
|
year: 'numeric',
|
|
month: 'short',
|
|
day: 'numeric',
|
|
hour: '2-digit',
|
|
minute: '2-digit',
|
|
second: '2-digit'
|
|
});
|
|
}
|
|
|
|
function main() {
|
|
console.log('🔍 Validating timestamp logic for backlog processing...\n');
|
|
|
|
const db = new Database(DB_PATH);
|
|
|
|
try {
|
|
// Check for pending messages
|
|
const pendingStats = db.query(`
|
|
SELECT
|
|
status,
|
|
COUNT(*) as count,
|
|
MIN(created_at_epoch) as earliest,
|
|
MAX(created_at_epoch) as latest
|
|
FROM pending_messages
|
|
GROUP BY status
|
|
ORDER BY status
|
|
`).all();
|
|
|
|
console.log('Pending Messages Status:\n');
|
|
for (const stat of pendingStats) {
|
|
console.log(`${stat.status}: ${stat.count} messages`);
|
|
if (stat.earliest && stat.latest) {
|
|
console.log(` Created: ${formatTimestamp(stat.earliest)} to ${formatTimestamp(stat.latest)}`);
|
|
}
|
|
}
|
|
console.log();
|
|
|
|
// Get sample pending messages with their session info
|
|
const pendingWithSessions = db.query(`
|
|
SELECT
|
|
pm.id,
|
|
pm.session_db_id,
|
|
pm.tool_name,
|
|
pm.created_at_epoch as msg_created,
|
|
pm.status,
|
|
s.sdk_session_id,
|
|
s.started_at_epoch as session_started,
|
|
s.project
|
|
FROM pending_messages pm
|
|
LEFT JOIN sdk_sessions s ON pm.session_db_id = s.id
|
|
WHERE pm.status IN ('pending', 'processing')
|
|
ORDER BY pm.created_at_epoch
|
|
LIMIT 10
|
|
`).all();
|
|
|
|
if (pendingWithSessions.length === 0) {
|
|
console.log('✅ No pending messages - all caught up!\n');
|
|
db.close();
|
|
return;
|
|
}
|
|
|
|
console.log(`Sample of ${pendingWithSessions.length} pending messages:\n`);
|
|
console.log('═══════════════════════════════════════════════════════════════════════');
|
|
|
|
for (const msg of pendingWithSessions) {
|
|
console.log(`\nPending Message #${msg.id}: ${msg.tool_name} (${msg.status})`);
|
|
console.log(` Created: ${formatTimestamp(msg.msg_created)}`);
|
|
|
|
if (msg.session_started) {
|
|
console.log(` Session started: ${formatTimestamp(msg.session_started)}`);
|
|
console.log(` Project: ${msg.project}`);
|
|
|
|
// Validate logic
|
|
const ageDays = Math.round((Date.now() - msg.msg_created) / (1000 * 60 * 60 * 24));
|
|
|
|
if (msg.msg_created < msg.session_started) {
|
|
console.log(` ⚠️ WARNING: Message created BEFORE session! This is impossible.`);
|
|
} else if (ageDays > 0) {
|
|
console.log(` 📅 Message is ${ageDays} days old`);
|
|
console.log(` ✅ Would use original timestamp: ${formatTimestamp(msg.msg_created)}`);
|
|
} else {
|
|
console.log(` ✅ Recent message, would use original timestamp: ${formatTimestamp(msg.msg_created)}`);
|
|
}
|
|
} else {
|
|
console.log(` ⚠️ No session found for session_db_id ${msg.session_db_id}`);
|
|
}
|
|
}
|
|
|
|
console.log('\n═══════════════════════════════════════════════════════════════════════');
|
|
console.log('\nTimestamp Logic Validation:\n');
|
|
console.log('✅ Code Flow:');
|
|
console.log(' 1. SessionManager.yieldNextMessage() tracks earliestPendingTimestamp');
|
|
console.log(' 2. SDKAgent captures originalTimestamp before processing');
|
|
console.log(' 3. processSDKResponse passes originalTimestamp to storeObservation/storeSummary');
|
|
console.log(' 4. SessionStore uses overrideTimestampEpoch ?? Date.now()');
|
|
console.log(' 5. earliestPendingTimestamp reset after batch completes\n');
|
|
|
|
console.log('✅ Expected Behavior:');
|
|
console.log(' - New messages: get current timestamp');
|
|
console.log(' - Backlog messages: get original created_at_epoch');
|
|
console.log(' - Observations match their source message timestamps\n');
|
|
|
|
// Check for any sessions with stuck processing messages
|
|
const stuckMessages = db.query(`
|
|
SELECT
|
|
session_db_id,
|
|
COUNT(*) as count,
|
|
MIN(created_at_epoch) as earliest,
|
|
MAX(created_at_epoch) as latest
|
|
FROM pending_messages
|
|
WHERE status = 'processing'
|
|
GROUP BY session_db_id
|
|
ORDER BY count DESC
|
|
`).all();
|
|
|
|
if (stuckMessages.length > 0) {
|
|
console.log('⚠️ Stuck Messages (status=processing):\n');
|
|
for (const stuck of stuckMessages) {
|
|
const ageDays = Math.round((Date.now() - stuck.earliest) / (1000 * 60 * 60 * 24));
|
|
console.log(` Session ${stuck.session_db_id}: ${stuck.count} messages`);
|
|
console.log(` Stuck for ${ageDays} days (${formatTimestamp(stuck.earliest)})`);
|
|
}
|
|
console.log('\n 💡 These will be processed with original timestamps when orphan processing is enabled\n');
|
|
}
|
|
|
|
} catch (error) {
|
|
console.error('❌ Error:', error);
|
|
process.exit(1);
|
|
} finally {
|
|
db.close();
|
|
}
|
|
}
|
|
|
|
main();
|