Performance improvements: Token reduction and enhanced summaries (#101)
* refactor: Reduce continuation prompt token usage by 95 lines Removed redundant instructions from continuation prompt that were originally added to mitigate a session continuity issue. That issue has since been resolved, making these detailed instructions unnecessary on every continuation. Changes: - Reduced continuation prompt from ~106 lines to ~11 lines (~95 line reduction) - Changed "User's Goal:" to "Next Prompt in Session:" (more accurate framing) - Removed redundant WHAT TO RECORD, WHEN TO SKIP, and OUTPUT FORMAT sections - Kept concise reminder: "Continue generating observations and progress summaries..." - Initial prompt still contains all detailed instructions Impact: - Significant token savings on every continuation prompt - Faster context injection with no loss of functionality - Instructions remain comprehensive in initial prompt Files modified: - src/sdk/prompts.ts (buildContinuationPrompt function) - plugin/scripts/worker-service.cjs (compiled output) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Enhance observation and summary prompts for clarity and token efficiency * Enhance prompt clarity and instructions in prompts.ts - Added a reminder to think about instructions before starting work. - Simplified the continuation prompt instruction by removing "for this ongoing session." * feat: Enhance settings.json with permissions and deny access to sensitive files refactor: Remove PLAN-full-observation-display.md and PR_SUMMARY.md as they are no longer needed chore: Delete SECURITY_SUMMARY.md since it is redundant after recent changes fix: Update worker-service.cjs to streamline observation generation instructions cleanup: Remove src-analysis.md and src-tree.md for a cleaner codebase refactor: Modify prompts.ts to clarify instructions for memory processing * refactor: Remove legacy worker service implementation * feat: Enhance summary hook to extract last assistant message and improve logging - Added function to extract the last assistant message from the transcript. - Updated summary hook to include last assistant message in the summary request. - Modified SDKSession interface to store last assistant message. - Adjusted buildSummaryPrompt to utilize last assistant message for generating summaries. - Updated worker service and session manager to handle last assistant message in summarize requests. - Introduced silentDebug utility for improved logging and diagnostics throughout the summary process. * docs: Add comprehensive implementation plan for ROI metrics feature Added detailed implementation plan covering: - Token usage capture from Agent SDK - Database schema changes (migration #8) - Discovery cost tracking per observation - Context hook display with ROI metrics - Testing and rollout strategy Timeline: ~20 hours over 4 days Goal: Empirical data for YC application amendment 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: Add transcript processing scripts for analysis and formatting - Implemented `dump-transcript-readable.ts` to generate a readable markdown dump of transcripts, excluding certain entry types. - Created `extract-rich-context-examples.ts` to extract and showcase rich context examples from transcripts, highlighting user requests and assistant reasoning. - Developed `format-transcript-context.ts` to format transcript context into a structured markdown format for improved observation generation. - Added `test-transcript-parser.ts` for validating data extraction from transcript JSONL files, including statistics and error reporting. - Introduced `transcript-to-markdown.ts` for a complete representation of transcript data in markdown format, showing all context data. - Enhanced type definitions in `transcript.ts` to support new features and ensure type safety. - Built `transcript-parser.ts` to handle parsing of transcript JSONL files, including error handling and data extraction methods. * Refactor hooks and SDKAgent for improved observation handling - Updated `new-hook.ts` to clean user prompts by stripping leading slashes for better semantic clarity. - Enhanced `save-hook.ts` to include additional tools in the SKIP_TOOLS set, preventing unnecessary observations from certain command invocations. - Modified `prompts.ts` to change the structure of observation prompts, emphasizing the observational role and providing a detailed XML output format for observations. - Adjusted `SDKAgent.ts` to enforce stricter tool usage restrictions, ensuring the memory agent operates solely as an observer without any tool access. * feat: Enhance session initialization to accept user prompts and prompt numbers - Updated `handleSessionInit` in `worker-service.ts` to extract `userPrompt` and `promptNumber` from the request body and pass them to `initializeSession`. - Modified `initializeSession` in `SessionManager.ts` to handle optional `currentUserPrompt` and `promptNumber` parameters. - Added logic to update the existing session's `userPrompt` and `lastPromptNumber` if a `currentUserPrompt` is provided. - Implemented debug logging for session initialization and updates to track user prompts and prompt numbers. --------- Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Alex Newman
@@ -0,0 +1,86 @@
|
||||
/**
|
||||
* Silent Debug Logger
|
||||
*
|
||||
* NOTE: This utility is to be used like Frank's Red Hot, we put that shit on everything.
|
||||
*
|
||||
* USE THIS INSTEAD OF SILENT FAILURES!
|
||||
* Stop doing this: `const value = something || '';`
|
||||
* Start doing this: `const value = something || silentDebug('something was undefined');`
|
||||
*
|
||||
* Logs to ~/.claude-mem/silent.log and returns a fallback value.
|
||||
* Check logs with `npm run logs:silent`
|
||||
*
|
||||
* Usage:
|
||||
* import { silentDebug } from '../utils/silent-debug.js';
|
||||
*
|
||||
* const title = obs.title || silentDebug('obs.title missing', { obs });
|
||||
* const name = user.name || silentDebug('user.name missing', { user }, 'Anonymous');
|
||||
*
|
||||
* try {
|
||||
* doSomething();
|
||||
* } catch (error) {
|
||||
* silentDebug('doSomething failed', { error });
|
||||
* }
|
||||
*/
|
||||
|
||||
import { appendFileSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
const LOG_FILE = join(homedir(), '.claude-mem', 'silent.log');
|
||||
|
||||
/**
|
||||
* Write a debug message to silent.log and return fallback value
|
||||
* @param message - The message to log
|
||||
* @param data - Optional data to include (will be JSON stringified)
|
||||
* @param fallback - Value to return (defaults to empty string)
|
||||
* @returns The fallback value (for use in || fallbacks)
|
||||
*/
|
||||
export function silentDebug(message: string, data?: any, fallback: string = ''): string {
|
||||
const timestamp = new Date().toISOString();
|
||||
|
||||
// Capture stack trace to get caller location
|
||||
const stack = new Error().stack || '';
|
||||
const stackLines = stack.split('\n');
|
||||
// Line 0: "Error"
|
||||
// Line 1: "at silentDebug ..."
|
||||
// Line 2: "at <CALLER> ..." <- We want this one
|
||||
const callerLine = stackLines[2] || '';
|
||||
const callerMatch = callerLine.match(/at\s+(?:.*\s+)?\(?([^:]+):(\d+):(\d+)\)?/);
|
||||
const location = callerMatch
|
||||
? `${callerMatch[1].split('/').pop()}:${callerMatch[2]}`
|
||||
: 'unknown';
|
||||
|
||||
let logLine = `[${timestamp}] [${location}] ${message}`;
|
||||
|
||||
if (data !== undefined) {
|
||||
try {
|
||||
logLine += ` ${JSON.stringify(data)}`;
|
||||
} catch (error) {
|
||||
logLine += ` [stringify error: ${error}]`;
|
||||
}
|
||||
}
|
||||
|
||||
logLine += '\n';
|
||||
|
||||
try {
|
||||
appendFileSync(LOG_FILE, logLine);
|
||||
} catch (error) {
|
||||
// If we can't write to the log file, fail silently (it's a debug utility after all)
|
||||
// Only write to stderr as a last resort
|
||||
console.error('[silent-debug] Failed to write to log:', error);
|
||||
}
|
||||
|
||||
return fallback;
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear the silent log file
|
||||
*/
|
||||
export function clearSilentLog(): void {
|
||||
try {
|
||||
appendFileSync(LOG_FILE, `\n${'='.repeat(80)}\n[${new Date().toISOString()}] Log cleared\n${'='.repeat(80)}\n\n`);
|
||||
} catch (error) {
|
||||
// Ignore errors
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,254 @@
|
||||
/**
|
||||
* TranscriptParser - Properly parse Claude Code transcript JSONL files
|
||||
* Handles all transcript entry types based on validated model
|
||||
*/
|
||||
|
||||
import { readFileSync } from 'fs';
|
||||
import type {
|
||||
TranscriptEntry,
|
||||
UserTranscriptEntry,
|
||||
AssistantTranscriptEntry,
|
||||
SummaryTranscriptEntry,
|
||||
SystemTranscriptEntry,
|
||||
QueueOperationTranscriptEntry,
|
||||
ContentItem,
|
||||
TextContent,
|
||||
} from '../types/transcript.js';
|
||||
|
||||
export interface ParseStats {
|
||||
totalLines: number;
|
||||
parsedEntries: number;
|
||||
failedLines: number;
|
||||
entriesByType: Record<string, number>;
|
||||
failureRate: number;
|
||||
}
|
||||
|
||||
export class TranscriptParser {
|
||||
private entries: TranscriptEntry[] = [];
|
||||
private parseErrors: Array<{ lineNumber: number; error: string }> = [];
|
||||
|
||||
constructor(transcriptPath: string) {
|
||||
this.parseTranscript(transcriptPath);
|
||||
}
|
||||
|
||||
private parseTranscript(transcriptPath: string): void {
|
||||
const content = readFileSync(transcriptPath, 'utf-8').trim();
|
||||
if (!content) return;
|
||||
|
||||
const lines = content.split('\n');
|
||||
|
||||
lines.forEach((line, index) => {
|
||||
try {
|
||||
const entry = JSON.parse(line) as TranscriptEntry;
|
||||
this.entries.push(entry);
|
||||
} catch (error) {
|
||||
this.parseErrors.push({
|
||||
lineNumber: index + 1,
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all entries of a specific type
|
||||
*/
|
||||
getEntriesByType<T extends TranscriptEntry>(type: T['type']): T[] {
|
||||
return this.entries.filter((e) => e.type === type) as T[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all user entries
|
||||
*/
|
||||
getUserEntries(): UserTranscriptEntry[] {
|
||||
return this.getEntriesByType<UserTranscriptEntry>('user');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all assistant entries
|
||||
*/
|
||||
getAssistantEntries(): AssistantTranscriptEntry[] {
|
||||
return this.getEntriesByType<AssistantTranscriptEntry>('assistant');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all summary entries
|
||||
*/
|
||||
getSummaryEntries(): SummaryTranscriptEntry[] {
|
||||
return this.getEntriesByType<SummaryTranscriptEntry>('summary');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all system entries
|
||||
*/
|
||||
getSystemEntries(): SystemTranscriptEntry[] {
|
||||
return this.getEntriesByType<SystemTranscriptEntry>('system');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all queue operation entries
|
||||
*/
|
||||
getQueueOperationEntries(): QueueOperationTranscriptEntry[] {
|
||||
return this.getEntriesByType<QueueOperationTranscriptEntry>('queue-operation');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get last entry of a specific type
|
||||
*/
|
||||
getLastEntryByType<T extends TranscriptEntry>(type: T['type']): T | null {
|
||||
const entries = this.getEntriesByType<T>(type);
|
||||
return entries.length > 0 ? entries[entries.length - 1] : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract text content from content items
|
||||
*/
|
||||
private extractTextFromContent(content: string | ContentItem[]): string {
|
||||
if (typeof content === 'string') {
|
||||
return content;
|
||||
}
|
||||
|
||||
if (Array.isArray(content)) {
|
||||
return content
|
||||
.filter((item): item is TextContent => item.type === 'text')
|
||||
.map((item) => item.text)
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
return '';
|
||||
}
|
||||
|
||||
/**
|
||||
* Get last user message text (finds last entry with actual text content)
|
||||
*/
|
||||
getLastUserMessage(): string {
|
||||
const userEntries = this.getUserEntries();
|
||||
|
||||
// Iterate backward to find the last user message with text content
|
||||
for (let i = userEntries.length - 1; i >= 0; i--) {
|
||||
const entry = userEntries[i];
|
||||
if (!entry?.message?.content) continue;
|
||||
|
||||
const text = this.extractTextFromContent(entry.message.content);
|
||||
if (text) return text;
|
||||
}
|
||||
|
||||
return '';
|
||||
}
|
||||
|
||||
/**
|
||||
* Get last assistant message text (finds last entry with text content, with optional system-reminder filtering)
|
||||
*/
|
||||
getLastAssistantMessage(filterSystemReminders = true): string {
|
||||
const assistantEntries = this.getAssistantEntries();
|
||||
|
||||
// Iterate backward to find the last assistant message with text content
|
||||
for (let i = assistantEntries.length - 1; i >= 0; i--) {
|
||||
const entry = assistantEntries[i];
|
||||
if (!entry?.message?.content) continue;
|
||||
|
||||
let text = this.extractTextFromContent(entry.message.content);
|
||||
if (!text) continue;
|
||||
|
||||
if (filterSystemReminders) {
|
||||
// Filter out system-reminder tags and their content
|
||||
text = text.replace(/<system-reminder>[\s\S]*?<\/system-reminder>/g, '');
|
||||
// Clean up excessive whitespace
|
||||
text = text.replace(/\n{3,}/g, '\n\n').trim();
|
||||
}
|
||||
|
||||
if (text) return text;
|
||||
}
|
||||
|
||||
return '';
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all tool use operations from assistant entries
|
||||
*/
|
||||
getToolUseHistory(): Array<{ name: string; timestamp: string; input: any }> {
|
||||
const toolUses: Array<{ name: string; timestamp: string; input: any }> = [];
|
||||
|
||||
for (const entry of this.getAssistantEntries()) {
|
||||
if (Array.isArray(entry.message.content)) {
|
||||
for (const item of entry.message.content) {
|
||||
if (item.type === 'tool_use') {
|
||||
toolUses.push({
|
||||
name: item.name,
|
||||
timestamp: entry.timestamp,
|
||||
input: item.input,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return toolUses;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get total token usage across all assistant messages
|
||||
*/
|
||||
getTotalTokenUsage(): {
|
||||
inputTokens: number;
|
||||
outputTokens: number;
|
||||
cacheCreationTokens: number;
|
||||
cacheReadTokens: number;
|
||||
} {
|
||||
const assistantEntries = this.getAssistantEntries();
|
||||
|
||||
return assistantEntries.reduce(
|
||||
(acc, entry) => {
|
||||
const usage = entry.message.usage;
|
||||
if (usage) {
|
||||
acc.inputTokens += usage.input_tokens || 0;
|
||||
acc.outputTokens += usage.output_tokens || 0;
|
||||
acc.cacheCreationTokens += usage.cache_creation_input_tokens || 0;
|
||||
acc.cacheReadTokens += usage.cache_read_input_tokens || 0;
|
||||
}
|
||||
return acc;
|
||||
},
|
||||
{
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
cacheCreationTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get parse statistics
|
||||
*/
|
||||
getParseStats(): ParseStats {
|
||||
const entriesByType: Record<string, number> = {};
|
||||
|
||||
for (const entry of this.entries) {
|
||||
entriesByType[entry.type] = (entriesByType[entry.type] || 0) + 1;
|
||||
}
|
||||
|
||||
const totalLines = this.entries.length + this.parseErrors.length;
|
||||
|
||||
return {
|
||||
totalLines,
|
||||
parsedEntries: this.entries.length,
|
||||
failedLines: this.parseErrors.length,
|
||||
entriesByType,
|
||||
failureRate: totalLines > 0 ? this.parseErrors.length / totalLines : 0,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get parse errors
|
||||
*/
|
||||
getParseErrors(): Array<{ lineNumber: number; error: string }> {
|
||||
return this.parseErrors;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all entries (raw)
|
||||
*/
|
||||
getAllEntries(): TranscriptEntry[] {
|
||||
return this.entries;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user