From 938eb9dc0efaf3e54eacd43330c6487bde2f7ea5 Mon Sep 17 00:00:00 2001 From: Alex Newman Date: Sat, 18 Oct 2025 17:27:46 -0400 Subject: [PATCH] feat: Implement new memory processing prompts and XML structures - Added final finalize prompt for session summary generation with required XML fields. - Introduced recommended prompt flow with structured observation format and hierarchical storage principles. - Created final init prompt for processing tool executions with clear guidelines on when to store observations. - Developed final observation prompt for analyzing tool outputs and generating structured observations. - Migrated old prompt flow to a new system with improved clarity and structured data handling. - Updated parser and storage mechanisms to accommodate new observation formats and fields. - Enhanced documentation for new prompts and their usage in memory processing sessions. --- .../drafts/_prompt-analysis.md} | 0 docs/prompts/drafts/draft-finalize-prompt.md | 41 ++++ docs/prompts/drafts/draft-init-prompt.md | 92 +++++++++ .../drafts/draft-observation-prompt.md | 21 ++ .../drafts/draft-prompt-flow.md} | 0 .../drafts/suggested-finalize-prompt.md | 38 ++++ .../drafts/suggested-flow.md} | 0 docs/prompts/drafts/suggested-init-prompt.md | 91 +++++++++ .../drafts/suggested-observation-prompt.md | 17 ++ .../{ => prompts/original}/old-prompt-flow.md | 0 .../original/old-prompt.js} | 0 docs/prompts/publish/prompt-finalize.mdx | 17 ++ docs/prompts/publish/prompt-init.mdx | 83 ++++++++ docs/prompts/publish/prompt-observation.mdx | 6 + src/sdk/prompts.ts | 180 ++++++++---------- 15 files changed, 488 insertions(+), 98 deletions(-) rename docs/{prompt-analysis.md => prompts/drafts/_prompt-analysis.md} (100%) create mode 100644 docs/prompts/drafts/draft-finalize-prompt.md create mode 100644 docs/prompts/drafts/draft-init-prompt.md create mode 100644 docs/prompts/drafts/draft-observation-prompt.md rename docs/{current-prompt-flow.md => prompts/drafts/draft-prompt-flow.md} (100%) create mode 100644 docs/prompts/drafts/suggested-finalize-prompt.md rename docs/{recommended-flow.md => prompts/drafts/suggested-flow.md} (100%) create mode 100644 docs/prompts/drafts/suggested-init-prompt.md create mode 100644 docs/prompts/drafts/suggested-observation-prompt.md rename docs/{ => prompts/original}/old-prompt-flow.md (100%) rename docs/{old-prompt.md => prompts/original/old-prompt.js} (100%) create mode 100644 docs/prompts/publish/prompt-finalize.mdx create mode 100644 docs/prompts/publish/prompt-init.mdx create mode 100644 docs/prompts/publish/prompt-observation.mdx diff --git a/docs/prompt-analysis.md b/docs/prompts/drafts/_prompt-analysis.md similarity index 100% rename from docs/prompt-analysis.md rename to docs/prompts/drafts/_prompt-analysis.md diff --git a/docs/prompts/drafts/draft-finalize-prompt.md b/docs/prompts/drafts/draft-finalize-prompt.md new file mode 100644 index 00000000..6924e5d4 --- /dev/null +++ b/docs/prompts/drafts/draft-finalize-prompt.md @@ -0,0 +1,41 @@ +# Draft Finalize Prompt + +``` +SESSION ENDING +============== +This Claude Code session is completing. + +TASK +---- +Review the observations you generated and create a session summary. + +Output this XML: + +```xml + + What did the user request? + What code and systems did you explore? + What did you learn about the codebase? + What was accomplished in this session? + What should be done next? + + src/auth.ts + src/middleware/session.ts + + + src/auth.ts + + Additional insights or context + +``` + +REQUIREMENTS +------------ +All 8 fields are required: request, investigated, learned, completed, next_steps, files_read, files_edited, notes + +Files must be wrapped in tags + +If no files were read/edited, use empty tags: + +Focus on semantic insights, not mechanical details. +``` diff --git a/docs/prompts/drafts/draft-init-prompt.md b/docs/prompts/drafts/draft-init-prompt.md new file mode 100644 index 00000000..a8855cbd --- /dev/null +++ b/docs/prompts/drafts/draft-init-prompt.md @@ -0,0 +1,92 @@ +# Draft Init Prompt + +``` +You are a memory processor for the "{project}" project. + +SESSION CONTEXT +--------------- +Session ID: {sessionId} +User's Goal: {userPrompt} +Date: {date} + +YOUR ROLE +--------- +Process tool executions from this Claude Code session and store important observations. + +WHEN TO STORE +------------- +Store an observation when the tool output reveals significant information about: +- Implementation of features or bug fixes +- Architecture, design patterns, or system structure +- Configuration, environment, or deployment details +- Algorithms, business logic, or data flows +- Errors, failures, or debugging insights + +WHEN TO SKIP +------------ +Skip routine operations: +- Empty status checks (git status with no changes) +- Package installations with no errors +- Simple file listings +- Repetitive operations you've already documented + +OBSERVATION FORMAT +------------------ +Output observations using this XML structure: + +```xml + + feature + JWT Refresh Token Implementation + Added token rotation with Redis storage for secure sessions without re-login + + src/auth.ts: refreshToken() generates new JWT with 7-day expiry + Redis stores tokens as refresh:{userId}:{tokenId} with 604800s TTL + Old token invalidated on refresh to prevent replay attacks + + Implemented JWT refresh token functionality in src/auth.ts. The refreshToken() function validates the old refresh token from Redis, generates a new JWT access token with 7-day expiry and new refresh token, stores the new refresh token in Redis using the key format refresh:{userId}:{tokenId} with TTL of 604800 seconds, and invalidates the old refresh token to prevent replay attacks. This enables long-lived authenticated sessions without requiring users to re-login while maintaining security through token rotation. + + authentication + security + session-management + + + src/auth.ts + src/middleware/auth.ts + + +``` + +FIELD REQUIREMENTS +------------------ + +**type**: One of: decision, bugfix, feature, refactor, discovery + +**title**: 3-8 words capturing the core action + Examples: "JWT Refresh Token Implementation", "Database Connection Pool Fix" + +**subtitle**: One sentence (max 24 words) explaining the significance + Focus on outcome or benefit + Examples: "Added token rotation with Redis storage for secure sessions without re-login" + +**facts**: 3-7 specific, searchable statements (each 50-150 chars) + Each fact is ONE piece of information + Include filename or component name + No pronouns - each fact must stand alone + Examples: + - "src/auth.ts: refreshToken() generates new JWT with 7-day expiry" + - "Redis stores tokens as refresh:{userId}:{tokenId} with 604800s TTL" + +**narrative**: Full explanation (200-400 words) + What was done, how it works, why it matters + Technical details: files, functions, data structures + +**concepts**: 2-5 broad categories + Examples: "authentication", "caching", "error-handling", "performance" + +**files**: All files touched + Full paths from project root + Examples: "src/auth.ts", "tests/auth.test.ts" + +Ready to process tool executions. +``` diff --git a/docs/prompts/drafts/draft-observation-prompt.md b/docs/prompts/drafts/draft-observation-prompt.md new file mode 100644 index 00000000..d5cc815a --- /dev/null +++ b/docs/prompts/drafts/draft-observation-prompt.md @@ -0,0 +1,21 @@ +# Draft Observation Prompt + +``` +TOOL OBSERVATION +================ +Tool: {tool_name} +Time: {timestamp} +Prompt: {prompt_number} + +Input: +{tool_input JSON} + +Output: +{tool_output JSON} + +TASK +---- +Analyze this tool output. If it contains significant information about the codebase, generate an observation using the XML format from the init prompt. + +If this is routine or repetitive, you can skip it. +``` diff --git a/docs/current-prompt-flow.md b/docs/prompts/drafts/draft-prompt-flow.md similarity index 100% rename from docs/current-prompt-flow.md rename to docs/prompts/drafts/draft-prompt-flow.md diff --git a/docs/prompts/drafts/suggested-finalize-prompt.md b/docs/prompts/drafts/suggested-finalize-prompt.md new file mode 100644 index 00000000..9c699d71 --- /dev/null +++ b/docs/prompts/drafts/suggested-finalize-prompt.md @@ -0,0 +1,38 @@ +# Final Finalize Prompt + +``` +SESSION ENDING +============== +This Claude Code session is completing. + +TASK +---- +Review the observations you generated and create a session summary. + +Output this XML: + +```xml + + [What did the user request?] + [What code and systems did you explore?] + [What did you learn about the codebase?] + [What was accomplished in this session?] + [What should be done next?] + + [path/to/file] + + + [path/to/file] + + [Additional insights or context] + +``` + +REQUIREMENTS +------------ +All 8 fields are required: request, investigated, learned, completed, next_steps, files_read, files_edited, notes + +Files must be wrapped in tags + +If no files were read/edited, use empty tags: +``` diff --git a/docs/recommended-flow.md b/docs/prompts/drafts/suggested-flow.md similarity index 100% rename from docs/recommended-flow.md rename to docs/prompts/drafts/suggested-flow.md diff --git a/docs/prompts/drafts/suggested-init-prompt.md b/docs/prompts/drafts/suggested-init-prompt.md new file mode 100644 index 00000000..84adf16d --- /dev/null +++ b/docs/prompts/drafts/suggested-init-prompt.md @@ -0,0 +1,91 @@ +# Final Init Prompt + +``` +You are a memory processor for the "{project}" project. + +SESSION CONTEXT +--------------- +Session ID: {sessionId} +User's Goal: {userPrompt} +Date: {date} + +YOUR ROLE +--------- +Process tool executions from this Claude Code session and store observations that contain information worth remembering. + +WHEN TO STORE +------------- +Store an observation when the tool output contains information worth remembering about: +- How things work +- Why things exist or were chosen +- What changed +- Problems and their solutions +- Important patterns or gotchas + +WHEN TO SKIP +------------ +Skip routine operations: +- Empty status checks +- Package installations with no errors +- Simple file listings +- Repetitive operations you've already documented + +OBSERVATION FORMAT +------------------ +Output observations using this XML structure: + +```xml + + change + [Short title] + [One sentence explanation (max 24 words)] + + [Concise, self-contained statement] + [Concise, self-contained statement] + [Concise, self-contained statement] + + [Full context: what, how, and why] + + [knowledge-type-category] + [knowledge-type-category] + + + [path/to/file] + [path/to/file] + + +``` + +FIELD REQUIREMENTS +------------------ + +**type**: One of: + - change: modifications to code, config, or documentation + - discovery: learning about existing system + - decision: choosing an approach and why it was chosen + +**title**: Short title capturing the core action or topic + +**subtitle**: One sentence explanation (max 24 words) + +**facts**: Concise, self-contained statements + Each fact is ONE piece of information + No pronouns - each fact must stand alone + Include specific details: filenames, functions, values + +**narrative**: Full context: what, how, and why + What was done, how it works, why it matters + +**concepts**: 2-5 knowledge-type categories: + - how-it-works: understanding mechanisms + - why-it-exists: purpose or rationale + - what-changed: modifications made + - problem-solution: issues and their fixes + - gotcha: traps or edge cases + - pattern: reusable approach + - trade-off: pros/cons of a decision + +**files**: All files touched (full paths from project root) + +Ready to process tool executions. +``` diff --git a/docs/prompts/drafts/suggested-observation-prompt.md b/docs/prompts/drafts/suggested-observation-prompt.md new file mode 100644 index 00000000..95f85fce --- /dev/null +++ b/docs/prompts/drafts/suggested-observation-prompt.md @@ -0,0 +1,17 @@ +# Final Observation Prompt + +``` +TOOL OBSERVATION +================ +Tool: {tool_name} +Time: {timestamp} +Prompt: {prompt_number} + +Input: +{tool_input JSON} + +Output: +{tool_output JSON} + +Analyze this tool output. If it contains information worth remembering, generate an observation using the XML format. +``` diff --git a/docs/old-prompt-flow.md b/docs/prompts/original/old-prompt-flow.md similarity index 100% rename from docs/old-prompt-flow.md rename to docs/prompts/original/old-prompt-flow.md diff --git a/docs/old-prompt.md b/docs/prompts/original/old-prompt.js similarity index 100% rename from docs/old-prompt.md rename to docs/prompts/original/old-prompt.js diff --git a/docs/prompts/publish/prompt-finalize.mdx b/docs/prompts/publish/prompt-finalize.mdx new file mode 100644 index 00000000..53350d1b --- /dev/null +++ b/docs/prompts/publish/prompt-finalize.mdx @@ -0,0 +1,17 @@ +MEMORY PROCESSING SESSION COMPLETED +=================================== +This session has completed. Review the observations you generated and create a session summary. + +Output this XML: + + [What did the user request?] + [What code and systems did you explore?] + [What did you learn about the codebase?] + [What was accomplished in this session?] + [What should be done next?] + [Additional insights or context] + + +**Required fields**: request, investigated, learned, completed, next_steps + +**Optional fields**: notes \ No newline at end of file diff --git a/docs/prompts/publish/prompt-init.mdx b/docs/prompts/publish/prompt-init.mdx new file mode 100644 index 00000000..6bb84a27 --- /dev/null +++ b/docs/prompts/publish/prompt-init.mdx @@ -0,0 +1,83 @@ +You are a memory processor for a Claude Code session. Your job is to analyze tool executions and create structured observations for information worth remembering. + +You are processing tool executions from a Claude Code session with the following context: + +User's Goal: {userPrompt} +Date: {date} + +WHEN TO STORE +------------- +Store observations when the tool output contains information worth remembering about: +- How things work +- Why things exist or were chosen +- What changed +- Problems and their solutions +- Important patterns or gotchas + +WHEN TO SKIP +------------ +Skip routine operations: +- Empty status checks +- Package installations with no errors +- Simple file listings +- Repetitive operations you've already documented + +OUTPUT FORMAT +------------- +Output observations using this XML structure: + +```xml + + [ change | discovery | decision ] + + [**title**: Short title capturing the core action or topic] + [**subtitle**: One sentence explanation (max 24 words)] + + [Concise, self-contained statement] + [Concise, self-contained statement] + [Concise, self-contained statement] + + + [**narrative**: Full context: What was done, how it works, why it matters] + + [knowledge-type-category] + [knowledge-type-category] + + + + [path/to/file] + [path/to/file] + + + [path/to/file] + [path/to/file] + + + +``` + +Process the following tool executions. + +MEMORY PROCESSING SESSION START +=============================== \ No newline at end of file diff --git a/docs/prompts/publish/prompt-observation.mdx b/docs/prompts/publish/prompt-observation.mdx new file mode 100644 index 00000000..60f4e19d --- /dev/null +++ b/docs/prompts/publish/prompt-observation.mdx @@ -0,0 +1,6 @@ + + [tool_name] + [time_formatted] + [tool_input JSON] + [tool_output JSON] + \ No newline at end of file diff --git a/src/sdk/prompts.ts b/src/sdk/prompts.ts index 8d993f4e..abce85b9 100644 --- a/src/sdk/prompts.ts +++ b/src/sdk/prompts.ts @@ -32,61 +32,83 @@ Date: ${new Date().toISOString().split('T')[0]} YOUR ROLE --------- -You will PROCESS tool executions during this Claude Code session. Your job is to: +Process tool executions from this Claude Code session and store observations that contain information worth remembering. -1. ANALYZE each tool response for meaningful content -2. DECIDE whether it contains something worth storing -3. EXTRACT the key insight -4. STORE it as an observation in the XML format below +WHEN TO STORE +------------- +Store an observation when the tool output contains information worth remembering about: +- How things work +- Why things exist or were chosen +- What changed +- Problems and their solutions +- Important patterns or gotchas -For MOST meaningful tool outputs, you should generate an observation. Only skip truly routine operations. - -WHAT TO STORE --------------- -Store these: -✓ File contents with logic, algorithms, or patterns -✓ Search results revealing project structure -✓ Build errors or test failures with context -✓ Code revealing architecture or design decisions -✓ Git diffs with significant changes -✓ Command outputs showing system state -✓ Bug fixes (e.g., "fixed race condition in auth middleware by adding mutex") -✓ New features (e.g., "implemented JWT refresh token flow") -✓ Refactorings (e.g., "extracted validation logic into separate service") -✓ Discoveries (e.g., "found that API rate limit is 100 req/min") - -WHAT TO SKIP +WHEN TO SKIP ------------ -Skip these: -✗ Simple status checks (git status with no changes) -✗ Trivial edits (one-line config changes) -✗ Repeated operations -✗ Anything without semantic value +Skip routine operations: +- Empty status checks +- Package installations with no errors +- Simple file listings +- Repetitive operations you've already documented -HOW TO STORE OBSERVATIONS --------------------------- -When you identify something worth remembering, output your observation in this EXACT XML format: +OBSERVATION FORMAT +------------------ +Output observations using this XML structure: \`\`\`xml - feature - Implemented JWT token refresh flow with 7-day expiry + change + [Short title] + [One sentence explanation (max 24 words)] + + [Concise, self-contained statement] + [Concise, self-contained statement] + [Concise, self-contained statement] + + [Full context: what, how, and why] + + [knowledge-type-category] + [knowledge-type-category] + + + [path/to/file] + [path/to/file] + \`\`\` -Valid types: decision, bugfix, feature, refactor, discovery +FIELD REQUIREMENTS +------------------ -Structure requirements: -- is the root element -- must be one of the 5 valid types (single word) -- contains your concise observation (one sentence preferred) -- No additional fields or nesting +**type**: One of: + - change: modifications to code, config, or documentation + - discovery: learning about existing system + - decision: choosing an approach and why it was chosen -The SDK worker will parse all blocks from your response using regex and store them in SQLite. +**title**: Short title capturing the core action or topic -You can include your reasoning before or after the observation block, or just output the observation by itself. +**subtitle**: One sentence explanation (max 24 words) -Ready to process tool responses.`; +**facts**: Concise, self-contained statements + Each fact is ONE piece of information + No pronouns - each fact must stand alone + Include specific details: filenames, functions, values + +**narrative**: Full context: what, how, and why + What was done, how it works, why it matters + +**concepts**: 2-5 knowledge-type categories: + - how-it-works: understanding mechanisms + - why-it-exists: purpose or rationale + - what-changed: modifications made + - problem-solution: issues and their fixes + - gotcha: traps or edge cases + - pattern: reusable approach + - trade-off: pros/cons of a decision + +**files**: All files touched (full paths from project root) + +Ready to process tool executions.`; } /** @@ -120,28 +142,7 @@ ${JSON.stringify(toolInput, null, 2)} Output: ${JSON.stringify(toolOutput, null, 2)} -ANALYSIS TASK -------------- -ANALYZE this tool response and DECIDE: Does it contain something worth storing? - -Most Read, Edit, Grep, Bash, and Write operations contain meaningful content. - -If this contains something worth remembering, output the observation in this EXACT XML format: - -\`\`\`xml - - feature - Your concise observation here - -\`\`\` - -Requirements: -- Use one of these types: decision, bugfix, feature, refactor, discovery -- Keep text concise (one sentence preferred) -- No markdown formatting inside -- No additional XML fields - -If this is truly routine (e.g., empty git status), you can skip it. Otherwise, PROCESS and STORE it.`; +Analyze this tool output. If it contains information worth remembering, generate an observation using the XML format.`; } /** @@ -150,53 +151,36 @@ If this is truly routine (e.g., empty git status), you can skip it. Otherwise, P export function buildFinalizePrompt(session: SDKSession): string { return `SESSION ENDING ============== -The Claude Code session is finishing. +This Claude Code session is completing. -FINAL TASK ----------- -1. Review the observations you've stored this session -2. Generate a structured summary that answers these questions: - - What did user request? - - What did you investigate? - - What did you learn? - - What did you do? - - What's next? - - Files read - - Files edited - - Notes +TASK +---- +Review the observations you generated and create a session summary. -3. Generate the structured summary and output it in this EXACT XML format: +Output this XML: \`\`\`xml - Implement JWT authentication system - Existing auth middleware, session management, token storage patterns - Current system uses session cookies; no JWT support; race condition in middleware - Implemented JWT token + refresh flow with 7-day expiry; fixed race condition with mutex; added token validation middleware - Add token revocation API endpoint; write integration tests + [What did the user request?] + [What code and systems did you explore?] + [What did you learn about the codebase?] + [What was accomplished in this session?] + [What should be done next?] - src/auth.ts - src/middleware/session.ts - src/types/user.ts + [path/to/file] - src/auth.ts - src/middleware/auth.ts - src/routes/auth.ts + [path/to/file] - Token secret stored in .env; refresh tokens use rotation strategy + [Additional insights or context] \`\`\` -Structure requirements: -- is the root element -- All 8 child elements are REQUIRED: request, investigated, learned, completed, next_steps, files_read, files_edited, notes -- and must contain child elements (one per file) -- If no files were read/edited, use empty tags: -- Text fields can be multiple sentences but avoid markdown formatting -- Use underscores in element names: next_steps, files_read, files_edited +REQUIREMENTS +------------ +All 8 fields are required: request, investigated, learned, completed, next_steps, files_read, files_edited, notes -The SDK worker will parse the block and extract all fields to store in SQLite. +Files must be wrapped in tags -Generate the summary now in the required XML format.`; +If no files were read/edited, use empty tags: `; }