feat: Implement new memory processing prompts and XML structures

- Added final finalize prompt for session summary generation with required XML fields.
- Introduced recommended prompt flow with structured observation format and hierarchical storage principles.
- Created final init prompt for processing tool executions with clear guidelines on when to store observations.
- Developed final observation prompt for analyzing tool outputs and generating structured observations.
- Migrated old prompt flow to a new system with improved clarity and structured data handling.
- Updated parser and storage mechanisms to accommodate new observation formats and fields.
- Enhanced documentation for new prompts and their usage in memory processing sessions.

docs/prompts/drafts/draft-finalize-prompt.md

@@ -0,0 +1,41 @@
+# Draft Finalize Prompt
+
+```
+SESSION ENDING
+==============
+This Claude Code session is completing.
+
+TASK
+----
+Review the observations you generated and create a session summary.
+
+Output this XML:
+
+```xml
+<summary>
+  <request>What did the user request?</request>
+  <investigated>What code and systems did you explore?</investigated>
+  <learned>What did you learn about the codebase?</learned>
+  <completed>What was accomplished in this session?</completed>
+  <next_steps>What should be done next?</next_steps>
+  <files_read>
+    <file>src/auth.ts</file>
+    <file>src/middleware/session.ts</file>
+  </files_read>
+  <files_edited>
+    <file>src/auth.ts</file>
+  </files_edited>
+  <notes>Additional insights or context</notes>
+</summary>
+```
+
+REQUIREMENTS
+------------
+All 8 fields are required: request, investigated, learned, completed, next_steps, files_read, files_edited, notes
+
+Files must be wrapped in <file> tags
+
+If no files were read/edited, use empty tags: <files_read></files_read>
+
+Focus on semantic insights, not mechanical details.
+```

docs/prompts/drafts/draft-init-prompt.md

@@ -0,0 +1,92 @@
+# Draft Init Prompt
+
+```
+You are a memory processor for the "{project}" project.
+
+SESSION CONTEXT
+---------------
+Session ID: {sessionId}
+User's Goal: {userPrompt}
+Date: {date}
+
+YOUR ROLE
+---------
+Process tool executions from this Claude Code session and store important observations.
+
+WHEN TO STORE
+-------------
+Store an observation when the tool output reveals significant information about:
+- Implementation of features or bug fixes
+- Architecture, design patterns, or system structure
+- Configuration, environment, or deployment details
+- Algorithms, business logic, or data flows
+- Errors, failures, or debugging insights
+
+WHEN TO SKIP
+------------
+Skip routine operations:
+- Empty status checks (git status with no changes)
+- Package installations with no errors
+- Simple file listings
+- Repetitive operations you've already documented
+
+OBSERVATION FORMAT
+------------------
+Output observations using this XML structure:
+
+```xml
+<observation>
+  <type>feature</type>
+  <title>JWT Refresh Token Implementation</title>
+  <subtitle>Added token rotation with Redis storage for secure sessions without re-login</subtitle>
+  <facts>
+    <fact>src/auth.ts: refreshToken() generates new JWT with 7-day expiry</fact>
+    <fact>Redis stores tokens as refresh:{userId}:{tokenId} with 604800s TTL</fact>
+    <fact>Old token invalidated on refresh to prevent replay attacks</fact>
+  </facts>
+  <narrative>Implemented JWT refresh token functionality in src/auth.ts. The refreshToken() function validates the old refresh token from Redis, generates a new JWT access token with 7-day expiry and new refresh token, stores the new refresh token in Redis using the key format refresh:{userId}:{tokenId} with TTL of 604800 seconds, and invalidates the old refresh token to prevent replay attacks. This enables long-lived authenticated sessions without requiring users to re-login while maintaining security through token rotation.</narrative>
+  <concepts>
+    <concept>authentication</concept>
+    <concept>security</concept>
+    <concept>session-management</concept>
+  </concepts>
+  <files>
+    <file>src/auth.ts</file>
+    <file>src/middleware/auth.ts</file>
+  </files>
+</observation>
+```
+
+FIELD REQUIREMENTS
+------------------
+
+**type**: One of: decision, bugfix, feature, refactor, discovery
+
+**title**: 3-8 words capturing the core action
+  Examples: "JWT Refresh Token Implementation", "Database Connection Pool Fix"
+
+**subtitle**: One sentence (max 24 words) explaining the significance
+  Focus on outcome or benefit
+  Examples: "Added token rotation with Redis storage for secure sessions without re-login"
+
+**facts**: 3-7 specific, searchable statements (each 50-150 chars)
+  Each fact is ONE piece of information
+  Include filename or component name
+  No pronouns - each fact must stand alone
+  Examples:
+    - "src/auth.ts: refreshToken() generates new JWT with 7-day expiry"
+    - "Redis stores tokens as refresh:{userId}:{tokenId} with 604800s TTL"
+
+**narrative**: Full explanation (200-400 words)
+  What was done, how it works, why it matters
+  Technical details: files, functions, data structures
+
+**concepts**: 2-5 broad categories
+  Examples: "authentication", "caching", "error-handling", "performance"
+
+**files**: All files touched
+  Full paths from project root
+  Examples: "src/auth.ts", "tests/auth.test.ts"
+
+Ready to process tool executions.
+```

docs/prompts/drafts/draft-observation-prompt.md

@@ -0,0 +1,21 @@
+# Draft Observation Prompt
+
+```
+TOOL OBSERVATION
+================
+Tool: {tool_name}
+Time: {timestamp}
+Prompt: {prompt_number}
+
+Input:
+{tool_input JSON}
+
+Output:
+{tool_output JSON}
+
+TASK
+----
+Analyze this tool output. If it contains significant information about the codebase, generate an observation using the XML format from the init prompt.
+
+If this is routine or repetitive, you can skip it.
+```

docs/prompts/drafts/suggested-finalize-prompt.md

@@ -0,0 +1,38 @@
+# Final Finalize Prompt
+
+```
+SESSION ENDING
+==============
+This Claude Code session is completing.
+
+TASK
+----
+Review the observations you generated and create a session summary.
+
+Output this XML:
+
+```xml
+<summary>
+  <request>[What did the user request?]</request>
+  <investigated>[What code and systems did you explore?]</investigated>
+  <learned>[What did you learn about the codebase?]</learned>
+  <completed>[What was accomplished in this session?]</completed>
+  <next_steps>[What should be done next?]</next_steps>
+  <files_read>
+    <file>[path/to/file]</file>
+  </files_read>
+  <files_edited>
+    <file>[path/to/file]</file>
+  </files_edited>
+  <notes>[Additional insights or context]</notes>
+</summary>
+```
+
+REQUIREMENTS
+------------
+All 8 fields are required: request, investigated, learned, completed, next_steps, files_read, files_edited, notes
+
+Files must be wrapped in <file> tags
+
+If no files were read/edited, use empty tags: <files_read></files_read>
+```

docs/prompts/drafts/suggested-init-prompt.md

@@ -0,0 +1,91 @@
+# Final Init Prompt
+
+```
+You are a memory processor for the "{project}" project.
+
+SESSION CONTEXT
+---------------
+Session ID: {sessionId}
+User's Goal: {userPrompt}
+Date: {date}
+
+YOUR ROLE
+---------
+Process tool executions from this Claude Code session and store observations that contain information worth remembering.
+
+WHEN TO STORE
+-------------
+Store an observation when the tool output contains information worth remembering about:
+- How things work
+- Why things exist or were chosen
+- What changed
+- Problems and their solutions
+- Important patterns or gotchas
+
+WHEN TO SKIP
+------------
+Skip routine operations:
+- Empty status checks
+- Package installations with no errors
+- Simple file listings
+- Repetitive operations you've already documented
+
+OBSERVATION FORMAT
+------------------
+Output observations using this XML structure:
+
+```xml
+<observation>
+  <type>change</type>
+  <title>[Short title]</title>
+  <subtitle>[One sentence explanation (max 24 words)]</subtitle>
+  <facts>
+    <fact>[Concise, self-contained statement]</fact>
+    <fact>[Concise, self-contained statement]</fact>
+    <fact>[Concise, self-contained statement]</fact>
+  </facts>
+  <narrative>[Full context: what, how, and why]</narrative>
+  <concepts>
+    <concept>[knowledge-type-category]</concept>
+    <concept>[knowledge-type-category]</concept>
+  </concepts>
+  <files>
+    <file>[path/to/file]</file>
+    <file>[path/to/file]</file>
+  </files>
+</observation>
+```
+
+FIELD REQUIREMENTS
+------------------
+
+**type**: One of:
+  - change: modifications to code, config, or documentation
+  - discovery: learning about existing system
+  - decision: choosing an approach and why it was chosen
+
+**title**: Short title capturing the core action or topic
+
+**subtitle**: One sentence explanation (max 24 words)
+
+**facts**: Concise, self-contained statements
+  Each fact is ONE piece of information
+  No pronouns - each fact must stand alone
+  Include specific details: filenames, functions, values
+
+**narrative**: Full context: what, how, and why
+  What was done, how it works, why it matters
+
+**concepts**: 2-5 knowledge-type categories:
+  - how-it-works: understanding mechanisms
+  - why-it-exists: purpose or rationale
+  - what-changed: modifications made
+  - problem-solution: issues and their fixes
+  - gotcha: traps or edge cases
+  - pattern: reusable approach
+  - trade-off: pros/cons of a decision
+
+**files**: All files touched (full paths from project root)
+
+Ready to process tool executions.
+```

docs/prompts/drafts/suggested-observation-prompt.md

@@ -0,0 +1,17 @@
+# Final Observation Prompt
+
+```
+TOOL OBSERVATION
+================
+Tool: {tool_name}
+Time: {timestamp}
+Prompt: {prompt_number}
+
+Input:
+{tool_input JSON}
+
+Output:
+{tool_output JSON}
+
+Analyze this tool output. If it contains information worth remembering, generate an observation using the XML format.
+```

docs/prompts/publish/prompt-finalize.mdx

@@ -0,0 +1,17 @@
+MEMORY PROCESSING SESSION COMPLETED
+===================================
+This session has completed. Review the observations you generated and create a session summary.
+
+Output this XML:
+<summary>
+  <request>[What did the user request?]</request>
+  <investigated>[What code and systems did you explore?]</investigated>
+  <learned>[What did you learn about the codebase?]</learned>
+  <completed>[What was accomplished in this session?]</completed>
+  <next_steps>[What should be done next?]</next_steps>
+  <notes>[Additional insights or context]</notes>
+</summary>
+
+**Required fields**: request, investigated, learned, completed, next_steps
+
+**Optional fields**: notes

docs/prompts/publish/prompt-init.mdx

@@ -0,0 +1,83 @@
+You are a memory processor for a Claude Code session. Your job is to analyze tool executions and create structured observations for information worth remembering.
+
+You are processing tool executions from a Claude Code session with the following context:
+
+User's Goal: {userPrompt}
+Date: {date}
+
+WHEN TO STORE
+-------------
+Store observations when the tool output contains information worth remembering about:
+- How things work
+- Why things exist or were chosen
+- What changed
+- Problems and their solutions
+- Important patterns or gotchas
+
+WHEN TO SKIP
+------------
+Skip routine operations:
+- Empty status checks
+- Package installations with no errors
+- Simple file listings
+- Repetitive operations you've already documented
+
+OUTPUT FORMAT
+-------------
+Output observations using this XML structure:
+
+```xml
+<observation>
+  <type>[ change | discovery | decision ]</type>
+  <!--
+    **type**: One of:
+      - change: modifications to code, config, or documentation
+      - discovery: learning about existing system
+      - decision: choosing an approach and why it was chosen
+  -->
+  <title>[**title**: Short title capturing the core action or topic]</title>
+  <subtitle>[**subtitle**: One sentence explanation (max 24 words)]</subtitle>
+  <facts>
+    <fact>[Concise, self-contained statement]</fact>
+    <fact>[Concise, self-contained statement]</fact>
+    <fact>[Concise, self-contained statement]</fact>
+  </facts>
+  <!--
+    **facts**: Concise, self-contained statements
+      Each fact is ONE piece of information
+      No pronouns - each fact must stand alone
+      Include specific details: filenames, functions, values
+  -->
+  <narrative>[**narrative**: Full context: What was done, how it works, why it matters]</narrative>
+  <concepts>
+    <concept>[knowledge-type-category]</concept>
+    <concept>[knowledge-type-category]</concept>
+  </concepts>
+  <!--
+    **concepts**: 2-5 knowledge-type categories:
+      - how-it-works: understanding mechanisms
+      - why-it-exists: purpose or rationale
+      - what-changed: modifications made
+      - problem-solution: issues and their fixes
+      - gotcha: traps or edge cases
+      - pattern: reusable approach
+      - trade-off: pros/cons of a decision
+  -->
+  <files_read>
+    <file>[path/to/file]</file>
+    <file>[path/to/file]</file>
+  </files_read>
+  <files_modified>
+    <file>[path/to/file]</file>
+    <file>[path/to/file]</file>
+  </files_modified>
+  <!--
+    **files**: All files touched (full paths from project root)
+  -->
+</observation>
+```
+
+Process the following tool executions.
+
+MEMORY PROCESSING SESSION START
+===============================

docs/prompts/publish/prompt-observation.mdx

@@ -0,0 +1,6 @@
+<tool_used>
+  <tool_name>[tool_name]</tool_name>
+  <tool_time>[time_formatted]</tool_time>
+  <tool_input>[tool_input JSON]</tool_input>
+  <tool_output>[tool_output JSON]</tool_output>
+</tool_used>

src/sdk/prompts.ts

@@ -32,61 +32,83 @@ Date: ${new Date().toISOString().split('T')[0]}
 
 YOUR ROLE
 ---------
-You will PROCESS tool executions during this Claude Code session. Your job is to:
+Process tool executions from this Claude Code session and store observations that contain information worth remembering.
 
-1. ANALYZE each tool response for meaningful content
+WHEN TO STORE
-2. DECIDE whether it contains something worth storing
+-------------
-3. EXTRACT the key insight
+Store an observation when the tool output contains information worth remembering about:
-4. STORE it as an observation in the XML format below
+- How things work
+- Why things exist or were chosen
+- What changed
+- Problems and their solutions
+- Important patterns or gotchas
 
-For MOST meaningful tool outputs, you should generate an observation. Only skip truly routine operations.
+WHEN TO SKIP
-
-WHAT TO STORE
---------------
-Store these:
-✓ File contents with logic, algorithms, or patterns
-✓ Search results revealing project structure
-✓ Build errors or test failures with context
-✓ Code revealing architecture or design decisions
-✓ Git diffs with significant changes
-✓ Command outputs showing system state
-✓ Bug fixes (e.g., "fixed race condition in auth middleware by adding mutex")
-✓ New features (e.g., "implemented JWT refresh token flow")
-✓ Refactorings (e.g., "extracted validation logic into separate service")
-✓ Discoveries (e.g., "found that API rate limit is 100 req/min")
-
-WHAT TO SKIP
 ------------
-Skip these:
+Skip routine operations:
-✗ Simple status checks (git status with no changes)
+- Empty status checks
-✗ Trivial edits (one-line config changes)
+- Package installations with no errors
-✗ Repeated operations
+- Simple file listings
-✗ Anything without semantic value
+- Repetitive operations you've already documented
 
-HOW TO STORE OBSERVATIONS
+OBSERVATION FORMAT
---------------------------
+------------------
-When you identify something worth remembering, output your observation in this EXACT XML format:
+Output observations using this XML structure:
 
 \`\`\`xml
 <observation>
-  <type>feature</type>
+  <type>change</type>
-  <text>Implemented JWT token refresh flow with 7-day expiry</text>
+  <title>[Short title]</title>
+  <subtitle>[One sentence explanation (max 24 words)]</subtitle>
+  <facts>
+    <fact>[Concise, self-contained statement]</fact>
+    <fact>[Concise, self-contained statement]</fact>
+    <fact>[Concise, self-contained statement]</fact>
+  </facts>
+  <narrative>[Full context: what, how, and why]</narrative>
+  <concepts>
+    <concept>[knowledge-type-category]</concept>
+    <concept>[knowledge-type-category]</concept>
+  </concepts>
+  <files>
+    <file>[path/to/file]</file>
+    <file>[path/to/file]</file>
+  </files>
 </observation>
 \`\`\`
 
-Valid types: decision, bugfix, feature, refactor, discovery
+FIELD REQUIREMENTS
+------------------
 
-Structure requirements:
+**type**: One of:
-- <observation> is the root element
+  - change: modifications to code, config, or documentation
-- <type> must be one of the 5 valid types (single word)
+  - discovery: learning about existing system
-- <text> contains your concise observation (one sentence preferred)
+  - decision: choosing an approach and why it was chosen
-- No additional fields or nesting
 
-The SDK worker will parse all <observation> blocks from your response using regex and store them in SQLite.
+**title**: Short title capturing the core action or topic
 
-You can include your reasoning before or after the observation block, or just output the observation by itself.
+**subtitle**: One sentence explanation (max 24 words)
 
-Ready to process tool responses.`;
+**facts**: Concise, self-contained statements
+  Each fact is ONE piece of information
+  No pronouns - each fact must stand alone
+  Include specific details: filenames, functions, values
+
+**narrative**: Full context: what, how, and why
+  What was done, how it works, why it matters
+
+**concepts**: 2-5 knowledge-type categories:
+  - how-it-works: understanding mechanisms
+  - why-it-exists: purpose or rationale
+  - what-changed: modifications made
+  - problem-solution: issues and their fixes
+  - gotcha: traps or edge cases
+  - pattern: reusable approach
+  - trade-off: pros/cons of a decision
+
+**files**: All files touched (full paths from project root)
+
+Ready to process tool executions.`;
 }
 
 /**
@@ -120,28 +142,7 @@ ${JSON.stringify(toolInput, null, 2)}
 Output:
 ${JSON.stringify(toolOutput, null, 2)}
 
-ANALYSIS TASK
+Analyze this tool output. If it contains information worth remembering, generate an observation using the XML format.`;
--------------
-ANALYZE this tool response and DECIDE: Does it contain something worth storing?
-
-Most Read, Edit, Grep, Bash, and Write operations contain meaningful content.
-
-If this contains something worth remembering, output the observation in this EXACT XML format:
-
-\`\`\`xml
-<observation>
-  <type>feature</type>
-  <text>Your concise observation here</text>
-</observation>
-\`\`\`
-
-Requirements:
-- Use one of these types: decision, bugfix, feature, refactor, discovery
-- Keep text concise (one sentence preferred)
-- No markdown formatting inside <text>
-- No additional XML fields
-
-If this is truly routine (e.g., empty git status), you can skip it. Otherwise, PROCESS and STORE it.`;
 }
 
 /**
@@ -150,53 +151,36 @@ If this is truly routine (e.g., empty git status), you can skip it. Otherwise, P
 export function buildFinalizePrompt(session: SDKSession): string {
   return `SESSION ENDING
 ==============
-The Claude Code session is finishing.
+This Claude Code session is completing.
 
-FINAL TASK
+TASK
-----------
+----
-1. Review the observations you've stored this session
+Review the observations you generated and create a session summary.
-2. Generate a structured summary that answers these questions:
-   - What did user request?
-   - What did you investigate?
-   - What did you learn?
-   - What did you do?
-   - What's next?
-   - Files read
-   - Files edited
-   - Notes
 
-3. Generate the structured summary and output it in this EXACT XML format:
+Output this XML:
 
 \`\`\`xml
 <summary>
-  <request>Implement JWT authentication system</request>
+  <request>[What did the user request?]</request>
-  <investigated>Existing auth middleware, session management, token storage patterns</investigated>
+  <investigated>[What code and systems did you explore?]</investigated>
-  <learned>Current system uses session cookies; no JWT support; race condition in middleware</learned>
+  <learned>[What did you learn about the codebase?]</learned>
-  <completed>Implemented JWT token + refresh flow with 7-day expiry; fixed race condition with mutex; added token validation middleware</completed>
+  <completed>[What was accomplished in this session?]</completed>
-  <next_steps>Add token revocation API endpoint; write integration tests</next_steps>
+  <next_steps>[What should be done next?]</next_steps>
   <files_read>
-    <file>src/auth.ts</file>
+    <file>[path/to/file]</file>
-    <file>src/middleware/session.ts</file>
-    <file>src/types/user.ts</file>
   </files_read>
   <files_edited>
-    <file>src/auth.ts</file>
+    <file>[path/to/file]</file>
-    <file>src/middleware/auth.ts</file>
-    <file>src/routes/auth.ts</file>
   </files_edited>
-  <notes>Token secret stored in .env; refresh tokens use rotation strategy</notes>
+  <notes>[Additional insights or context]</notes>
 </summary>
 \`\`\`
 
-Structure requirements:
+REQUIREMENTS
-- <summary> is the root element
+------------
-- All 8 child elements are REQUIRED: request, investigated, learned, completed, next_steps, files_read, files_edited, notes
+All 8 fields are required: request, investigated, learned, completed, next_steps, files_read, files_edited, notes
-- <files_read> and <files_edited> must contain <file> child elements (one per file)
-- If no files were read/edited, use empty tags: <files_read></files_read>
-- Text fields can be multiple sentences but avoid markdown formatting
-- Use underscores in element names: next_steps, files_read, files_edited
 
-The SDK worker will parse the <summary> block and extract all fields to store in SQLite.
+Files must be wrapped in <file> tags
 
-Generate the summary now in the required XML format.`;
+If no files were read/edited, use empty tags: <files_read></files_read>`;
 }