Files
claude-mem/src/types/transcript.ts
T
Alex Newman 68290a9121 Performance improvements: Token reduction and enhanced summaries (#101)
* refactor: Reduce continuation prompt token usage by 95 lines

Removed redundant instructions from continuation prompt that were originally
added to mitigate a session continuity issue. That issue has since been
resolved, making these detailed instructions unnecessary on every continuation.

Changes:
- Reduced continuation prompt from ~106 lines to ~11 lines (~95 line reduction)
- Changed "User's Goal:" to "Next Prompt in Session:" (more accurate framing)
- Removed redundant WHAT TO RECORD, WHEN TO SKIP, and OUTPUT FORMAT sections
- Kept concise reminder: "Continue generating observations and progress summaries..."
- Initial prompt still contains all detailed instructions

Impact:
- Significant token savings on every continuation prompt
- Faster context injection with no loss of functionality
- Instructions remain comprehensive in initial prompt

Files modified:
- src/sdk/prompts.ts (buildContinuationPrompt function)
- plugin/scripts/worker-service.cjs (compiled output)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Enhance observation and summary prompts for clarity and token efficiency

* Enhance prompt clarity and instructions in prompts.ts

- Added a reminder to think about instructions before starting work.
- Simplified the continuation prompt instruction by removing "for this ongoing session."

* feat: Enhance settings.json with permissions and deny access to sensitive files

refactor: Remove PLAN-full-observation-display.md and PR_SUMMARY.md as they are no longer needed

chore: Delete SECURITY_SUMMARY.md since it is redundant after recent changes

fix: Update worker-service.cjs to streamline observation generation instructions

cleanup: Remove src-analysis.md and src-tree.md for a cleaner codebase

refactor: Modify prompts.ts to clarify instructions for memory processing

* refactor: Remove legacy worker service implementation

* feat: Enhance summary hook to extract last assistant message and improve logging

- Added function to extract the last assistant message from the transcript.
- Updated summary hook to include last assistant message in the summary request.
- Modified SDKSession interface to store last assistant message.
- Adjusted buildSummaryPrompt to utilize last assistant message for generating summaries.
- Updated worker service and session manager to handle last assistant message in summarize requests.
- Introduced silentDebug utility for improved logging and diagnostics throughout the summary process.

* docs: Add comprehensive implementation plan for ROI metrics feature

Added detailed implementation plan covering:
- Token usage capture from Agent SDK
- Database schema changes (migration #8)
- Discovery cost tracking per observation
- Context hook display with ROI metrics
- Testing and rollout strategy

Timeline: ~20 hours over 4 days
Goal: Empirical data for YC application amendment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add transcript processing scripts for analysis and formatting

- Implemented `dump-transcript-readable.ts` to generate a readable markdown dump of transcripts, excluding certain entry types.
- Created `extract-rich-context-examples.ts` to extract and showcase rich context examples from transcripts, highlighting user requests and assistant reasoning.
- Developed `format-transcript-context.ts` to format transcript context into a structured markdown format for improved observation generation.
- Added `test-transcript-parser.ts` for validating data extraction from transcript JSONL files, including statistics and error reporting.
- Introduced `transcript-to-markdown.ts` for a complete representation of transcript data in markdown format, showing all context data.
- Enhanced type definitions in `transcript.ts` to support new features and ensure type safety.
- Built `transcript-parser.ts` to handle parsing of transcript JSONL files, including error handling and data extraction methods.

* Refactor hooks and SDKAgent for improved observation handling

- Updated `new-hook.ts` to clean user prompts by stripping leading slashes for better semantic clarity.
- Enhanced `save-hook.ts` to include additional tools in the SKIP_TOOLS set, preventing unnecessary observations from certain command invocations.
- Modified `prompts.ts` to change the structure of observation prompts, emphasizing the observational role and providing a detailed XML output format for observations.
- Adjusted `SDKAgent.ts` to enforce stricter tool usage restrictions, ensuring the memory agent operates solely as an observer without any tool access.

* feat: Enhance session initialization to accept user prompts and prompt numbers

- Updated `handleSessionInit` in `worker-service.ts` to extract `userPrompt` and `promptNumber` from the request body and pass them to `initializeSession`.
- Modified `initializeSession` in `SessionManager.ts` to handle optional `currentUserPrompt` and `promptNumber` parameters.
- Added logic to update the existing session's `userPrompt` and `lastPromptNumber` if a `currentUserPrompt` is provided.
- Implemented debug logging for session initialization and updates to track user prompts and prompt numbers.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-13 18:22:44 -05:00

175 lines
3.4 KiB
TypeScript

/**
* TypeScript types for Claude Code transcript JSONL structure
* Based on Python Pydantic models from docs/context/cc-transcript-model-example.py
*/
export interface TodoItem {
id: string;
content: string;
status: 'pending' | 'in_progress' | 'completed';
priority: 'high' | 'medium' | 'low';
}
export interface UsageInfo {
input_tokens?: number;
cache_creation_input_tokens?: number;
cache_read_input_tokens?: number;
output_tokens?: number;
service_tier?: string;
server_tool_use?: any;
}
export interface TextContent {
type: 'text';
text: string;
}
export interface ToolUseContent {
type: 'tool_use';
id: string;
name: string;
input: Record<string, any>;
}
export interface ToolResultContent {
type: 'tool_result';
tool_use_id: string;
content: string | Array<Record<string, any>>;
is_error?: boolean;
}
export interface ThinkingContent {
type: 'thinking';
thinking: string;
signature?: string;
}
export interface ImageSource {
type: 'base64';
media_type: string;
data: string;
}
export interface ImageContent {
type: 'image';
source: ImageSource;
}
export type ContentItem =
| TextContent
| ToolUseContent
| ToolResultContent
| ThinkingContent
| ImageContent;
export interface UserMessage {
role: 'user';
content: string | ContentItem[];
}
export interface AssistantMessage {
id: string;
type: 'message';
role: 'assistant';
model: string;
content: ContentItem[];
stop_reason?: string;
stop_sequence?: string;
usage?: UsageInfo;
}
export interface FileInfo {
filePath: string;
content: string;
numLines: number;
startLine: number;
totalLines: number;
}
export interface FileReadResult {
type: 'text';
file: FileInfo;
}
export interface CommandResult {
stdout: string;
stderr: string;
interrupted: boolean;
isImage: boolean;
}
export interface TodoResult {
oldTodos: TodoItem[];
newTodos: TodoItem[];
}
export interface EditResult {
oldString?: string;
newString?: string;
replaceAll?: boolean;
originalFile?: string;
structuredPatch?: any;
userModified?: boolean;
}
export type ToolUseResult =
| string
| TodoItem[]
| FileReadResult
| CommandResult
| TodoResult
| EditResult
| ContentItem[];
export interface BaseTranscriptEntry {
parentUuid?: string;
isSidechain: boolean;
userType: string;
cwd: string;
sessionId: string;
version: string;
uuid: string;
timestamp: string;
isMeta?: boolean;
}
export interface UserTranscriptEntry extends BaseTranscriptEntry {
type: 'user';
message: UserMessage;
toolUseResult?: ToolUseResult;
}
export interface AssistantTranscriptEntry extends BaseTranscriptEntry {
type: 'assistant';
message: AssistantMessage;
requestId?: string;
}
export interface SummaryTranscriptEntry {
type: 'summary';
summary: string;
leafUuid: string;
cwd?: string;
}
export interface SystemTranscriptEntry extends BaseTranscriptEntry {
type: 'system';
content: string;
level?: string; // 'warning', 'info', 'error'
}
export interface QueueOperationTranscriptEntry {
type: 'queue-operation';
operation: 'enqueue' | 'dequeue';
timestamp: string;
sessionId: string;
content?: ContentItem[]; // Only present for enqueue operations
}
export type TranscriptEntry =
| UserTranscriptEntry
| AssistantTranscriptEntry
| SummaryTranscriptEntry
| SystemTranscriptEntry
| QueueOperationTranscriptEntry;