Performance improvements: Token reduction and enhanced summaries (#101)
* refactor: Reduce continuation prompt token usage by 95 lines Removed redundant instructions from continuation prompt that were originally added to mitigate a session continuity issue. That issue has since been resolved, making these detailed instructions unnecessary on every continuation. Changes: - Reduced continuation prompt from ~106 lines to ~11 lines (~95 line reduction) - Changed "User's Goal:" to "Next Prompt in Session:" (more accurate framing) - Removed redundant WHAT TO RECORD, WHEN TO SKIP, and OUTPUT FORMAT sections - Kept concise reminder: "Continue generating observations and progress summaries..." - Initial prompt still contains all detailed instructions Impact: - Significant token savings on every continuation prompt - Faster context injection with no loss of functionality - Instructions remain comprehensive in initial prompt Files modified: - src/sdk/prompts.ts (buildContinuationPrompt function) - plugin/scripts/worker-service.cjs (compiled output) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Enhance observation and summary prompts for clarity and token efficiency * Enhance prompt clarity and instructions in prompts.ts - Added a reminder to think about instructions before starting work. - Simplified the continuation prompt instruction by removing "for this ongoing session." * feat: Enhance settings.json with permissions and deny access to sensitive files refactor: Remove PLAN-full-observation-display.md and PR_SUMMARY.md as they are no longer needed chore: Delete SECURITY_SUMMARY.md since it is redundant after recent changes fix: Update worker-service.cjs to streamline observation generation instructions cleanup: Remove src-analysis.md and src-tree.md for a cleaner codebase refactor: Modify prompts.ts to clarify instructions for memory processing * refactor: Remove legacy worker service implementation * feat: Enhance summary hook to extract last assistant message and improve logging - Added function to extract the last assistant message from the transcript. - Updated summary hook to include last assistant message in the summary request. - Modified SDKSession interface to store last assistant message. - Adjusted buildSummaryPrompt to utilize last assistant message for generating summaries. - Updated worker service and session manager to handle last assistant message in summarize requests. - Introduced silentDebug utility for improved logging and diagnostics throughout the summary process. * docs: Add comprehensive implementation plan for ROI metrics feature Added detailed implementation plan covering: - Token usage capture from Agent SDK - Database schema changes (migration #8) - Discovery cost tracking per observation - Context hook display with ROI metrics - Testing and rollout strategy Timeline: ~20 hours over 4 days Goal: Empirical data for YC application amendment 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: Add transcript processing scripts for analysis and formatting - Implemented `dump-transcript-readable.ts` to generate a readable markdown dump of transcripts, excluding certain entry types. - Created `extract-rich-context-examples.ts` to extract and showcase rich context examples from transcripts, highlighting user requests and assistant reasoning. - Developed `format-transcript-context.ts` to format transcript context into a structured markdown format for improved observation generation. - Added `test-transcript-parser.ts` for validating data extraction from transcript JSONL files, including statistics and error reporting. - Introduced `transcript-to-markdown.ts` for a complete representation of transcript data in markdown format, showing all context data. - Enhanced type definitions in `transcript.ts` to support new features and ensure type safety. - Built `transcript-parser.ts` to handle parsing of transcript JSONL files, including error handling and data extraction methods. * Refactor hooks and SDKAgent for improved observation handling - Updated `new-hook.ts` to clean user prompts by stripping leading slashes for better semantic clarity. - Enhanced `save-hook.ts` to include additional tools in the SKIP_TOOLS set, preventing unnecessary observations from certain command invocations. - Modified `prompts.ts` to change the structure of observation prompts, emphasizing the observational role and providing a detailed XML output format for observations. - Adjusted `SDKAgent.ts` to enforce stricter tool usage restrictions, ensuring the memory agent operates solely as an observer without any tool access. * feat: Enhance session initialization to accept user prompts and prompt numbers - Updated `handleSessionInit` in `worker-service.ts` to extract `userPrompt` and `promptNumber` from the request body and pass them to `initializeSession`. - Modified `initializeSession` in `SessionManager.ts` to handle optional `currentUserPrompt` and `promptNumber` parameters. - Added logic to update the existing session's `userPrompt` and `lastPromptNumber` if a `currentUserPrompt` is provided. - Implemented debug logging for session initialization and updates to track user prompts and prompt numbers. --------- Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Alex Newman
@@ -15,6 +15,7 @@ import { existsSync, readFileSync } from 'fs';
|
||||
import { DatabaseManager } from './DatabaseManager.js';
|
||||
import { SessionManager } from './SessionManager.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { silentDebug } from '../../utils/silent-debug.js';
|
||||
import { parseObservations, parseSummary } from '../../sdk/parser.js';
|
||||
import { buildInitPrompt, buildObservationPrompt, buildSummaryPrompt, buildContinuationPrompt } from '../../sdk/prompts.js';
|
||||
import type { ActiveSession, SDKUserMessage, PendingMessage } from '../worker-types.js';
|
||||
@@ -43,7 +44,21 @@ export class SDKAgent {
|
||||
|
||||
// Get model ID and disallowed tools
|
||||
const modelId = this.getModelId();
|
||||
const disallowedTools = ['Bash']; // Prevent infinite loops
|
||||
// Memory agent is OBSERVER ONLY - no tools allowed
|
||||
const disallowedTools = [
|
||||
'Bash', // Prevent infinite loops
|
||||
'Read', // No file reading
|
||||
'Write', // No file writing
|
||||
'Edit', // No file editing
|
||||
'Grep', // No code searching
|
||||
'Glob', // No file pattern matching
|
||||
'WebFetch', // No web fetching
|
||||
'WebSearch', // No web searching
|
||||
'Task', // No spawning sub-agents
|
||||
'NotebookEdit', // No notebook editing
|
||||
'AskUserQuestion',// No asking questions
|
||||
'TodoWrite' // No todo management
|
||||
];
|
||||
|
||||
// Create message generator (event-driven)
|
||||
const messageGenerator = this.createMessageGenerator(session);
|
||||
@@ -189,7 +204,8 @@ export class SDKAgent {
|
||||
sdk_session_id: session.sdkSessionId,
|
||||
project: session.project,
|
||||
user_prompt: session.userPrompt,
|
||||
last_user_message: message.last_user_message || ''
|
||||
last_user_message: message.last_user_message || '',
|
||||
last_assistant_message: message.last_assistant_message || ''
|
||||
})
|
||||
},
|
||||
session_id: session.claudeSessionId,
|
||||
@@ -221,15 +237,16 @@ export class SDKAgent {
|
||||
sessionId: session.sessionDbId,
|
||||
obsId,
|
||||
type: obs.type,
|
||||
title: obs.title.substring(0, 60) + (obs.title.length > 60 ? '...' : ''),
|
||||
files: obs.files?.length || 0,
|
||||
concepts: obs.concepts?.length || 0
|
||||
title: obs.title || silentDebug('obs.title is null', { obsId, type: obs.type }, '(untitled)'),
|
||||
filesRead: obs.files_read?.length ?? (silentDebug('obs.files_read is null/undefined', { obsId }), 0),
|
||||
filesModified: obs.files_modified?.length ?? (silentDebug('obs.files_modified is null/undefined', { obsId }), 0),
|
||||
concepts: obs.concepts?.length ?? (silentDebug('obs.concepts is null/undefined', { obsId }), 0)
|
||||
});
|
||||
|
||||
// Sync to Chroma with error logging
|
||||
const chromaStart = Date.now();
|
||||
const obsType = obs.type;
|
||||
const obsTitle = obs.title;
|
||||
const obsTitle = obs.title || silentDebug('obs.title is null for Chroma sync', { obsId, type: obs.type }, '(untitled)');
|
||||
this.dbManager.getChromaSync().syncObservation(
|
||||
obsId,
|
||||
session.claudeSessionId,
|
||||
@@ -239,21 +256,18 @@ export class SDKAgent {
|
||||
createdAtEpoch
|
||||
).then(() => {
|
||||
const chromaDuration = Date.now() - chromaStart;
|
||||
const truncatedTitle = obsTitle.length > 50
|
||||
? obsTitle.substring(0, 50) + '...'
|
||||
: obsTitle;
|
||||
logger.debug('CHROMA', 'Observation synced', {
|
||||
obsId,
|
||||
duration: `${chromaDuration}ms`,
|
||||
type: obsType,
|
||||
title: truncatedTitle
|
||||
title: obsTitle
|
||||
});
|
||||
}).catch(err => {
|
||||
logger.error('CHROMA', 'Failed to sync observation', {
|
||||
obsId,
|
||||
sessionId: session.sessionDbId,
|
||||
type: obsType,
|
||||
title: obsTitle.substring(0, 50)
|
||||
title: obsTitle
|
||||
}, err);
|
||||
});
|
||||
|
||||
@@ -298,14 +312,14 @@ export class SDKAgent {
|
||||
logger.info('SDK', 'Summary saved', {
|
||||
sessionId: session.sessionDbId,
|
||||
summaryId,
|
||||
request: summary.request.substring(0, 60) + (summary.request.length > 60 ? '...' : ''),
|
||||
request: summary.request || silentDebug('summary.request is null', { summaryId }, '(no request)'),
|
||||
hasCompleted: !!summary.completed,
|
||||
hasNextSteps: !!summary.next_steps
|
||||
});
|
||||
|
||||
// Sync to Chroma with error logging
|
||||
const chromaStart = Date.now();
|
||||
const summaryRequest = summary.request;
|
||||
const summaryRequest = summary.request || silentDebug('summary.request is null for Chroma sync', { summaryId }, '(no request)');
|
||||
this.dbManager.getChromaSync().syncSummary(
|
||||
summaryId,
|
||||
session.claudeSessionId,
|
||||
@@ -315,19 +329,16 @@ export class SDKAgent {
|
||||
createdAtEpoch
|
||||
).then(() => {
|
||||
const chromaDuration = Date.now() - chromaStart;
|
||||
const truncatedRequest = summaryRequest.length > 50
|
||||
? summaryRequest.substring(0, 50) + '...'
|
||||
: summaryRequest;
|
||||
logger.debug('CHROMA', 'Summary synced', {
|
||||
summaryId,
|
||||
duration: `${chromaDuration}ms`,
|
||||
request: truncatedRequest
|
||||
request: summaryRequest
|
||||
});
|
||||
}).catch(err => {
|
||||
logger.error('CHROMA', 'Failed to sync summary', {
|
||||
summaryId,
|
||||
sessionId: session.sessionDbId,
|
||||
request: summaryRequest.substring(0, 50)
|
||||
request: summaryRequest
|
||||
}, err);
|
||||
});
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
import { EventEmitter } from 'events';
|
||||
import { DatabaseManager } from './DatabaseManager.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { silentDebug } from '../../utils/silent-debug.js';
|
||||
import type { ActiveSession, PendingMessage, ObservationData } from '../worker-types.js';
|
||||
|
||||
export class SessionManager {
|
||||
@@ -33,27 +34,61 @@ export class SessionManager {
|
||||
/**
|
||||
* Initialize a new session or return existing one
|
||||
*/
|
||||
initializeSession(sessionDbId: number): ActiveSession {
|
||||
initializeSession(sessionDbId: number, currentUserPrompt?: string, promptNumber?: number): ActiveSession {
|
||||
// Check if already active
|
||||
let session = this.sessions.get(sessionDbId);
|
||||
if (session) {
|
||||
// Update userPrompt for continuation prompts
|
||||
if (currentUserPrompt) {
|
||||
silentDebug('[SessionManager] Updating userPrompt for continuation', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
oldPrompt: session.userPrompt.substring(0, 80),
|
||||
newPrompt: currentUserPrompt.substring(0, 80)
|
||||
});
|
||||
session.userPrompt = currentUserPrompt;
|
||||
session.lastPromptNumber = promptNumber || session.lastPromptNumber;
|
||||
} else {
|
||||
silentDebug('[SessionManager] No currentUserPrompt provided for existing session', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
usingCachedPrompt: session.userPrompt.substring(0, 80)
|
||||
});
|
||||
}
|
||||
return session;
|
||||
}
|
||||
|
||||
// Fetch from database
|
||||
const dbSession = this.dbManager.getSessionById(sessionDbId);
|
||||
|
||||
// Use currentUserPrompt if provided, otherwise fall back to database (first prompt)
|
||||
const userPrompt = currentUserPrompt || dbSession.user_prompt;
|
||||
|
||||
if (!currentUserPrompt) {
|
||||
silentDebug('[SessionManager] No currentUserPrompt provided for new session, using database', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
dbPrompt: dbSession.user_prompt.substring(0, 80)
|
||||
});
|
||||
} else {
|
||||
silentDebug('[SessionManager] Initializing session with fresh userPrompt', {
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
userPrompt: currentUserPrompt.substring(0, 80)
|
||||
});
|
||||
}
|
||||
|
||||
// Create active session
|
||||
session = {
|
||||
sessionDbId,
|
||||
claudeSessionId: dbSession.claude_session_id,
|
||||
sdkSessionId: null,
|
||||
project: dbSession.project,
|
||||
userPrompt: dbSession.user_prompt,
|
||||
userPrompt,
|
||||
pendingMessages: [],
|
||||
abortController: new AbortController(),
|
||||
generatorPromise: null,
|
||||
lastPromptNumber: this.dbManager.getSessionStore().getPromptCounter(sessionDbId),
|
||||
lastPromptNumber: promptNumber || this.dbManager.getSessionStore().getPromptCounter(sessionDbId),
|
||||
startTime: Date.now()
|
||||
};
|
||||
|
||||
@@ -123,7 +158,7 @@ export class SessionManager {
|
||||
* Queue a summarize request (zero-latency notification)
|
||||
* Auto-initializes session if not in memory but exists in database
|
||||
*/
|
||||
queueSummarize(sessionDbId: number, lastUserMessage: string): void {
|
||||
queueSummarize(sessionDbId: number, lastUserMessage: string, lastAssistantMessage?: string): void {
|
||||
// Auto-initialize from database if needed (handles worker restarts)
|
||||
let session = this.sessions.get(sessionDbId);
|
||||
if (!session) {
|
||||
@@ -134,7 +169,8 @@ export class SessionManager {
|
||||
|
||||
session.pendingMessages.push({
|
||||
type: 'summarize',
|
||||
last_user_message: lastUserMessage
|
||||
last_user_message: lastUserMessage,
|
||||
last_assistant_message: lastAssistantMessage
|
||||
});
|
||||
|
||||
const afterDepth = session.pendingMessages.length;
|
||||
|
||||
Reference in New Issue
Block a user