Compare commits
37 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 8b4cc4f6bf | |||
| 68290a9121 | |||
| ab5d78717f | |||
| cb4aea57a8 | |||
| 7bdf6dbfe1 | |||
| 1c9da73d5f | |||
| 5f7aa0710e | |||
| 39fedfc5fc | |||
| ecb8b39f6d | |||
| fe0902b48f | |||
| 4ab9739e4c | |||
| 6ddef1093a | |||
| 97d565e3cd | |||
| eafdd6a7be | |||
| 3529f9274b | |||
| 30ebe92a53 | |||
| 1bb203cbb5 | |||
| 9f8499fe54 | |||
| 40d105d7de | |||
| cda12d95c9 | |||
| bbd6113f69 | |||
| 51d1315562 | |||
| 85763d575d | |||
| c76ddc2f83 | |||
| 348cc7f7ac | |||
| ed19e92f75 | |||
| 2a96214456 | |||
| eaa2268bf9 | |||
| 730c420a13 | |||
| d0bdc6ae9b | |||
| d46fbb7166 | |||
| a60ef6eacb | |||
| 13941a1e72 | |||
| bd809c860e | |||
| b4b90faa1e | |||
| 9e235b5b57 | |||
| 5fdf25d60f |
@@ -10,7 +10,7 @@
|
||||
"plugins": [
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "5.4.0",
|
||||
"version": "6.0.0",
|
||||
"source": "./plugin",
|
||||
"description": "Persistent memory system for Claude Code - context compression across sessions"
|
||||
}
|
||||
|
||||
@@ -1,3 +1,14 @@
|
||||
{
|
||||
"env": {}
|
||||
"env": {},
|
||||
"permissions": {
|
||||
"deny": [
|
||||
"Read(./CHANGELOG.md)",
|
||||
"Read(./README.md)",
|
||||
"Read(./package-lock.json)",
|
||||
"Read(./docs/public/**)",
|
||||
"Read(./plugin/**)",
|
||||
"Read(./node_modules/**)",
|
||||
"Read(./.DS_Store)"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: version-bump
|
||||
description: Manage semantic version updates for claude-mem project. Handles patch, minor, and major version increments following semantic versioning. Updates package.json, marketplace.json, plugin.json, and CLAUDE.md version number (NOT version history). Creates git tags.
|
||||
description: Manage semantic version updates for claude-mem project. Handles patch, minor, and major version increments following semantic versioning. Updates package.json, marketplace.json, plugin.json, and CLAUDE.md version number (NOT version history). Creates git tags and GitHub releases. Auto-generates CHANGELOG.md from releases.
|
||||
---
|
||||
|
||||
# Version Bump Skill
|
||||
@@ -42,6 +42,7 @@ See [operations/workflow.md](operations/workflow.md) for detailed step-by-step p
|
||||
6. Build and test
|
||||
7. Commit and create git tag
|
||||
8. Push and create GitHub release
|
||||
9. Generate CHANGELOG.md from releases and commit
|
||||
|
||||
## Common Scenarios
|
||||
|
||||
@@ -56,6 +57,7 @@ See [operations/scenarios.md](operations/scenarios.md) for examples:
|
||||
- Update ALL FOUR files with matching version numbers
|
||||
- Create git tag with format `vX.Y.Z`
|
||||
- Create GitHub release from the tag
|
||||
- Generate CHANGELOG.md from releases after creating release
|
||||
- Ask user if version type is unclear
|
||||
|
||||
**NEVER:**
|
||||
@@ -73,6 +75,7 @@ Before considering the task complete:
|
||||
- [ ] Git tag created (format: vX.Y.Z)
|
||||
- [ ] Commit and tags pushed to remote
|
||||
- [ ] GitHub release created from the tag
|
||||
- [ ] CHANGELOG.md generated and committed
|
||||
- [ ] CLAUDE.md: ONLY line 9 updated (version number), NOT version history
|
||||
|
||||
## Reference Commands
|
||||
|
||||
@@ -187,6 +187,29 @@ gh release create vX.Y.Z --title "vX.Y.Z" --generate-notes
|
||||
|
||||
**IMPORTANT:** Always create the GitHub release immediately after pushing the tag. This makes the release discoverable to users and triggers any automated workflows.
|
||||
|
||||
## Step 10: Generate CHANGELOG
|
||||
|
||||
After creating the GitHub release, regenerate CHANGELOG.md from all releases:
|
||||
|
||||
```bash
|
||||
# Generate CHANGELOG.md from all GitHub releases
|
||||
npm run changelog:generate
|
||||
|
||||
# Review the generated changelog
|
||||
git diff CHANGELOG.md
|
||||
|
||||
# Commit and push the updated changelog
|
||||
git add CHANGELOG.md
|
||||
git commit -m "Update CHANGELOG.md for vX.Y.Z release"
|
||||
git push
|
||||
```
|
||||
|
||||
**Why this step:**
|
||||
- CHANGELOG.md is auto-generated from GitHub releases
|
||||
- Keeps the changelog in sync with release notes
|
||||
- No manual editing required
|
||||
- Single source of truth: GitHub releases
|
||||
|
||||
## Verification
|
||||
|
||||
After completing all steps, verify:
|
||||
|
||||
@@ -6,13 +6,17 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
|
||||
|
||||
**Your Role**: You are working on the plugin itself. When users interact with Claude Code with this plugin installed, your observations get captured and become their persistent memory.
|
||||
|
||||
**Current Version**: 5.4.0
|
||||
**Current Version**: 6.0.0
|
||||
|
||||
## IMPORTANT: Skills Are Auto-Invoked
|
||||
|
||||
**There is no `/skill` command.** Skills auto-invoke based on description metadata matching user queries. Don't document manual invocation (e.g., "Run `/skill troubleshoot`"). Instead: "The troubleshoot skill auto-activates when issues are detected."
|
||||
|
||||
## Critical Architecture Knowledge
|
||||
|
||||
### The Lifecycle Flow
|
||||
|
||||
1. **SessionStart** → `context-hook.ts` runs
|
||||
1. **SessionStart** → smart-install.js runs first (pre-hook), then `context-hook.ts` runs
|
||||
- Smart installer checks dependencies (cached, only runs on version changes)
|
||||
- Starts PM2 worker if not healthy
|
||||
- Injects context from previous sessions (configurable observation count)
|
||||
@@ -31,6 +35,8 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
|
||||
- Marks session complete (graceful, not DELETE)
|
||||
- Skips on `/clear` to preserve ongoing sessions
|
||||
|
||||
**Note**: smart-install.js is a pre-hook script (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json and only runs when dependencies need updating.
|
||||
|
||||
### Key Components
|
||||
|
||||
**Hooks** (`src/hooks/*.ts`)
|
||||
@@ -50,7 +56,7 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
|
||||
- FTS5 virtual tables for full-text search
|
||||
- `SessionStore` = CRUD, `SessionSearch` = FTS5 queries
|
||||
|
||||
**Search Skill** (`plugin/skills/search/SKILL.md`)
|
||||
**Search Skill** (`plugin/skills/mem-search/SKILL.md`)
|
||||
- Provides access to all search functionality via HTTP API + skill
|
||||
- Auto-invoked when users ask about past work, decisions, or history
|
||||
- Uses HTTP endpoints instead of MCP tools (~2,250 token savings per session)
|
||||
@@ -106,135 +112,25 @@ Changes to React components, styles, or viewer logic require rebuilding and rest
|
||||
2. `npm run sync-marketplace` → Syncs to `~/.claude/plugins/marketplaces/thedotmack/`
|
||||
3. Changes are live for next session (hooks/skills) or after restart (worker)
|
||||
|
||||
## Coding Standards: DRY, YAGNI, and Anti-Patterns
|
||||
## Coding Standards
|
||||
|
||||
**Philosophy**: Write the dumb, obvious thing first. Add complexity only when you actually hit the problem.
|
||||
**Philosophy**: Write the dumb, obvious thing first. Add complexity only when you hit the problem.
|
||||
|
||||
### Common Anti-Patterns to Avoid
|
||||
|
||||
**1. Wrapper Functions for Constants**
|
||||
```typescript
|
||||
// ❌ DON'T: Ceremonial wrapper that adds zero value
|
||||
export function getWorkerPort(): number {
|
||||
return FIXED_PORT;
|
||||
}
|
||||
|
||||
// ✅ DO: Export the constant directly
|
||||
export const WORKER_PORT = parseInt(process.env.CLAUDE_MEM_WORKER_PORT || "37777", 10);
|
||||
```
|
||||
|
||||
**2. Unused Default Parameters**
|
||||
```typescript
|
||||
// ❌ DON'T: Defaults that are never actually used
|
||||
async function isHealthy(timeout: number = 3000) { ... }
|
||||
// Every call: isHealthy(1000) - the default is dead code
|
||||
|
||||
// ✅ DO: Remove the default if no one uses it
|
||||
async function isHealthy(timeout: number) { ... }
|
||||
```
|
||||
|
||||
**3. Magic Numbers Everywhere**
|
||||
```typescript
|
||||
// ❌ DON'T: Unexplained magic numbers scattered throughout
|
||||
if (await isWorkerHealthy(1000)) { ... }
|
||||
await waitForHealth(10000);
|
||||
setTimeout(resolve, 100);
|
||||
|
||||
// ✅ DO: Named constants with context
|
||||
const HEALTH_CHECK_TIMEOUT_MS = 1000;
|
||||
const HEALTH_CHECK_MAX_WAIT_MS = 10000;
|
||||
const HEALTH_CHECK_POLL_INTERVAL_MS = 100;
|
||||
```
|
||||
|
||||
**4. Overengineered Error Handling**
|
||||
```typescript
|
||||
// ❌ DON'T: Silent failures and defensive programming for ghosts
|
||||
checkProcess.on("close", (code) => {
|
||||
// PM2 list can fail, but we should still continue - just assume worker isn't running
|
||||
resolve(); // <- Silent failure!
|
||||
});
|
||||
|
||||
// ✅ DO: Fail fast with clear errors
|
||||
checkProcess.on("close", (code) => {
|
||||
if (code !== 0) {
|
||||
reject(new Error(`PM2 not found - install dependencies first`));
|
||||
}
|
||||
resolve();
|
||||
});
|
||||
```
|
||||
|
||||
**5. Fragile String Parsing**
|
||||
```typescript
|
||||
// ❌ DON'T: Parse human-readable output with string matching
|
||||
const isRunning = output.includes("claude-mem-worker") && output.includes("online");
|
||||
|
||||
// ✅ DO: Use structured output (JSON)
|
||||
const processes = JSON.parse(execSync('pm2 jlist'));
|
||||
const isRunning = processes.some(p => p.name === 'claude-mem-worker' && p.pm2_env.status === 'online');
|
||||
```
|
||||
|
||||
**6. Duplicated Promise Wrappers**
|
||||
```typescript
|
||||
// ❌ DON'T: Copy-paste the same promise pattern multiple times
|
||||
await new Promise((resolve, reject) => {
|
||||
process1.on("error", reject);
|
||||
process1.on("close", (code) => { /* ... */ });
|
||||
});
|
||||
// ... later ...
|
||||
await new Promise((resolve, reject) => {
|
||||
process2.on("error", reject);
|
||||
process2.on("close", (code) => { /* ... same pattern */ });
|
||||
});
|
||||
|
||||
// ✅ DO: Extract a helper function
|
||||
async function waitForProcess(process: ChildProcess, validateExitCode = false): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
process.on("error", reject);
|
||||
process.on("close", (code) => {
|
||||
if (validateExitCode && code !== 0 && code !== null) {
|
||||
reject(new Error(`Process failed with exit code ${code}`));
|
||||
} else {
|
||||
resolve();
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
**7. YAGNI Violations - Solving Problems You Don't Have**
|
||||
```typescript
|
||||
// ❌ DON'T: 50+ lines checking PM2 status before starting
|
||||
const checkProcess = spawn(pm2Path, ["list", "--no-color"]);
|
||||
// ... parse output ...
|
||||
// ... check if running ...
|
||||
// ... then maybe start it ...
|
||||
|
||||
// ✅ DO: Just start it (PM2 start is idempotent)
|
||||
if (!await isWorkerHealthy()) {
|
||||
await startWorker(); // PM2 handles "already running" gracefully
|
||||
if (!await waitForWorkerHealth()) {
|
||||
throw new Error("Worker failed to become healthy");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Why These Patterns Appear
|
||||
|
||||
These anti-patterns often emerge from:
|
||||
- **Training bias**: Code that looks "professional" is often overengineered
|
||||
- **Risk aversion**: Optimizing for "what could go wrong" instead of "what do you actually need"
|
||||
- **Pattern matching**: Seeing a problem and immediately scaffolding a framework
|
||||
- **No real-world pain**: Not debugging at 2am means not feeling the cost of complexity
|
||||
|
||||
### The Actual Standard
|
||||
|
||||
1. **YAGNI (You Aren't Gonna Need It)**: Don't build it until you need it
|
||||
2. **DRY (Don't Repeat Yourself)**: Extract patterns after the second duplication, not before
|
||||
**Key Principles:**
|
||||
1. **YAGNI**: Don't build it until you need it
|
||||
2. **DRY**: Extract patterns after second duplication, not before
|
||||
3. **Fail Fast**: Explicit errors beat silent failures
|
||||
4. **Simple First**: Write the obvious solution, then optimize only if needed
|
||||
4. **Simple First**: Write the obvious solution, optimize only if needed
|
||||
5. **Delete Aggressively**: Less code = fewer bugs
|
||||
|
||||
**Reference**: See worker-utils.ts critique (conversation 2025-11-05) for detailed examples.
|
||||
**Common anti-patterns to avoid:**
|
||||
- Ceremonial wrapper functions for constants (just export the constant)
|
||||
- Unused default parameters (remove if never used)
|
||||
- Magic numbers without named constants
|
||||
- Silent failures instead of explicit errors
|
||||
- Fragile string parsing (use structured JSON output)
|
||||
- Copy-pasted promise wrappers (extract helper functions)
|
||||
- Overengineered "defensive" code for problems you don't have
|
||||
|
||||
## Common Tasks
|
||||
|
||||
@@ -244,6 +140,8 @@ These anti-patterns often emerge from:
|
||||
3. Add configuration to `plugin/hooks/hooks.json`
|
||||
4. Build and sync: `npm run build && npm run sync-marketplace`
|
||||
|
||||
**Note**: smart-install.js is not a hook - it's a pre-hook dependency checker that runs before context-hook via command chaining.
|
||||
|
||||
### Modifying Database Schema
|
||||
1. Update schema in `src/services/sqlite/schema.ts`
|
||||
2. Update SessionStore/SessionSearch classes
|
||||
@@ -263,175 +161,24 @@ pm2 delete claude-mem-worker # Force clean start
|
||||
2. `npm run build && npm run sync-marketplace`
|
||||
3. Start new Claude Code session (hooks) or restart worker (worker changes)
|
||||
4. Check `~/.claude-mem/claude-mem.db` for database state
|
||||
5. Use search skill to verify behavior (auto-invoked when asking about past work)
|
||||
5. Use mem-search skill to verify behavior (auto-invoked when asking about past work)
|
||||
|
||||
### Version Bumps
|
||||
Use the version-bump skill:
|
||||
```bash
|
||||
/skill version-bump
|
||||
```
|
||||
Choose patch/minor/major. Updates package.json, marketplace.json, plugin.json, and CLAUDE.md.
|
||||
Use the `version-bump` skill (auto-invokes when requesting version updates). It handles:
|
||||
- Semantic version increments (patch/minor/major)
|
||||
- Updates all version references (package.json, plugin.json, CLAUDE.md, marketplace.json)
|
||||
- Creates git tags and GitHub releases
|
||||
- Auto-generates CHANGELOG.md from releases
|
||||
|
||||
## Investigation Best Practices
|
||||
|
||||
**When investigations are failing persistently**, use Task agents for comprehensive file analysis instead of grep/search:
|
||||
When investigations fail persistently, use Task agents for comprehensive file analysis instead of repeated grep/search. Deploy agents to read full files and answer specific questions - more efficient than multiple rounds of searching.
|
||||
|
||||
**❌ Don't:** Repeatedly grep and search for patterns when failing to find the issue
|
||||
## Environment Variables
|
||||
|
||||
**✅ Do:** Deploy a Task agent to read files in full and answer specific questions
|
||||
```
|
||||
"Read these files in full and answer: [specific questions about the implementation]"
|
||||
- Reduces token usage by delegating to a specialized agent
|
||||
- Provides comprehensive analysis in one pass
|
||||
- Finds issues that grep might miss due to poor query formulation
|
||||
- More efficient than multiple rounds of searching
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Deploy a general-purpose Task agent to:
|
||||
1. Read src/hooks/context-hook.ts in full
|
||||
2. Read src/services/worker-service.ts in full
|
||||
3. Answer: How do these files work together? What's the current implementation state?
|
||||
4. Find any bugs or inconsistencies between them
|
||||
```
|
||||
|
||||
Use this when:
|
||||
- Investigating how multiple files interact
|
||||
- Search queries aren't finding what you expect
|
||||
- Need complete implementation context
|
||||
- Issue might be a subtle inconsistency between files
|
||||
|
||||
## Recent Changes
|
||||
|
||||
### v5.4.0 - Skill-Based Search Migration
|
||||
**Breaking Change**: MCP search tools replaced with skill-based approach
|
||||
- **Token Savings**: ~2,250 tokens per session start
|
||||
- **Progressive Disclosure**: Skill frontmatter (~250 tokens) instead of 9 MCP tool definitions (~2,500 tokens)
|
||||
- **New HTTP API**: 10 search endpoints in worker service (localhost:37777/api/search/*)
|
||||
- **Search Skill**: Auto-invoked when users ask about past work, decisions, or history
|
||||
- **No User Action Required**: Migration is transparent, searches work automatically
|
||||
- **Deprecated**: MCP search server (source kept for reference: src/servers/search-server.ts)
|
||||
|
||||
**Available Search Operations:**
|
||||
1. Search observations (full-text)
|
||||
2. Search session summaries (full-text)
|
||||
3. Search user prompts (full-text)
|
||||
4. Search by observation type (bugfix, feature, refactor, discovery, decision)
|
||||
5. Search by concept tag
|
||||
6. Search by file path
|
||||
7. Get recent context for a project
|
||||
8. Get timeline around specific point in time
|
||||
9. Get timeline by query (search + timeline in one call)
|
||||
10. Get API help documentation
|
||||
|
||||
**How It Works:**
|
||||
- User asks: "What bug did we fix last session?"
|
||||
- Claude sees skill description matches → invokes search skill
|
||||
- Skill loads full instructions → uses curl to call HTTP API → formats results
|
||||
- User sees formatted answer with past work context
|
||||
|
||||
### v5.1.2 - Theme Toggle
|
||||
**Theme Support**: Light/dark mode for viewer UI
|
||||
- User-selectable theme with persistent settings
|
||||
- Automatic system preference detection
|
||||
- Smooth transitions between themes
|
||||
- Settings stored in browser localStorage
|
||||
|
||||
### v5.1.0 - Web-Based Viewer UI
|
||||
**Major Feature**: Web-Based Viewer UI for Real-Time Memory Stream
|
||||
- Production-ready viewer accessible at http://localhost:37777
|
||||
- Real-time visualization via Server-Sent Events (SSE) - see observations, sessions, and prompts as they happen
|
||||
- Infinite scroll pagination with automatic deduplication
|
||||
- Project filtering to focus on specific codebases
|
||||
- Settings persistence (sidebar state, selected project)
|
||||
- Auto-reconnection with exponential backoff
|
||||
- GPU-accelerated animations for smooth interactions
|
||||
|
||||
**Worker Service API Endpoints** (14 HTTP/SSE endpoints total):
|
||||
|
||||
*Viewer & Health:*
|
||||
- `GET /` - Serves viewer HTML (self-contained React app)
|
||||
- `GET /health` - Health check endpoint
|
||||
- `GET /stream` - Server-Sent Events for real-time updates
|
||||
|
||||
*Data Retrieval:*
|
||||
- `GET /api/prompts` - Paginated user prompts with project filtering
|
||||
- `GET /api/observations` - Paginated observations with project filtering
|
||||
- `GET /api/summaries` - Paginated session summaries with project filtering
|
||||
- `GET /api/stats` - Database statistics (total counts by project)
|
||||
|
||||
*Settings:*
|
||||
- `GET /api/settings` - Get current viewer settings
|
||||
- `POST /api/settings` - Update viewer settings
|
||||
|
||||
*Session Management:*
|
||||
- `POST /sessions/:sessionDbId/init` - Initialize new session
|
||||
- `POST /sessions/:sessionDbId/observations` - Add observations to session
|
||||
- `POST /sessions/:sessionDbId/summarize` - Generate session summary
|
||||
- `GET /sessions/:sessionDbId/status` - Get session status
|
||||
- `DELETE /sessions/:sessionDbId` - Delete session (graceful cleanup)
|
||||
|
||||
**Database Enhancements** (+98 lines in SessionStore):
|
||||
- `getRecentPrompts()` - Paginated prompts with OFFSET/LIMIT
|
||||
- `getRecentObservations()` - Paginated observations with OFFSET/LIMIT
|
||||
- `getRecentSummaries()` - Paginated summaries with OFFSET/LIMIT
|
||||
- `getStats()` - Aggregated statistics by project
|
||||
- `getUniqueProjects()` - Distinct project names
|
||||
|
||||
**Complete React UI** (17 new files, 1,500+ lines):
|
||||
- Components: Header, Sidebar, Feed, Cards (Observation, Prompt, Summary, Skeleton)
|
||||
- Hooks: useSSE, usePagination, useSettings, useStats
|
||||
- Utils: Data merging, formatters, constants
|
||||
- Assets: Monaspace Radon font, logos (dark mode + logomark)
|
||||
- Build: esbuild pipeline for self-contained HTML bundle
|
||||
|
||||
**Why This Matters**: Users can now visualize their memory stream in real-time. See exactly what claude-mem is capturing as you work, filter by project, and understand the context being injected into sessions.
|
||||
|
||||
### v5.0.3 - Smart Install Caching
|
||||
**Smart Caching Installer for Windows Compatibility**:
|
||||
- Eliminated redundant npm install on every SessionStart (2-5s → 10ms)
|
||||
- Caches version in `.install-version` file
|
||||
- Only runs npm install when actually needed (first time, version change, missing deps)
|
||||
- 200x performance improvement for cached installations
|
||||
|
||||
### v5.0.0 - Hybrid Search Architecture
|
||||
**Major Feature**: Chroma Vector Database Integration
|
||||
- Hybrid semantic + keyword search combining ChromaDB with SQLite FTS5
|
||||
- ChromaSync service for automatic vector embedding synchronization (738 lines)
|
||||
- 90-day recency filtering for contextually relevant results
|
||||
- Timeline and context search capabilities (now provided via skill-based HTTP API)
|
||||
- Performance: Semantic search <200ms with 8,000+ vector documents
|
||||
- Full-text search across observations, sessions, and prompts
|
||||
|
||||
## Configuration Users Can Set
|
||||
|
||||
**Model Selection** (`~/.claude/settings.json`):
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"CLAUDE_MEM_MODEL": "claude-haiku-4-5" // or sonnet-4-5, opus-4, etc.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Context Observation Count** (`~/.claude/settings.json`):
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"CLAUDE_MEM_CONTEXT_OBSERVATIONS": "50" // default, adjust based on needs
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Worker Port** (`~/.claude/settings.json`):
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"CLAUDE_MEM_WORKER_PORT": "37777" // default
|
||||
}
|
||||
}
|
||||
```
|
||||
- `CLAUDE_MEM_MODEL` - Model for observations/summaries (default: claude-haiku-4-5)
|
||||
- `CLAUDE_MEM_CONTEXT_OBSERVATIONS` - Observations injected at SessionStart (default: 50)
|
||||
- `CLAUDE_MEM_WORKER_PORT` - Worker service port (default: 37777)
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
@@ -441,13 +188,13 @@ Hooks have strict timeout limits. PM2 manages a persistent background worker, al
|
||||
### Why SQLite FTS5
|
||||
Enables instant full-text search across thousands of observations without external dependencies. Automatic sync triggers keep FTS5 tables synchronized.
|
||||
|
||||
### Why Graceful Cleanup (v4.1.0)
|
||||
### Why Graceful Cleanup
|
||||
Changed from aggressive DELETE requests to marking sessions complete. Prevents interrupting summary generation and other async operations.
|
||||
|
||||
### Why Smart Install Caching (v5.0.3)
|
||||
### Why Smart Install Caching
|
||||
npm install is expensive (2-5s). Caching version state and only installing on changes makes SessionStart nearly instant (10ms).
|
||||
|
||||
### Why Web-Based Viewer UI (v5.1.0)
|
||||
### Why Web-Based Viewer UI
|
||||
Real-time visibility into memory stream helps users understand what's being captured and how context is being built. SSE provides instant updates without polling. Self-contained HTML bundle (esbuild) eliminates deployment complexity - everything served from a single file.
|
||||
|
||||
## File Locations
|
||||
@@ -465,6 +212,5 @@ Real-time visibility into memory stream helps users understand what's being capt
|
||||
**Sync**: `npm run sync-marketplace`
|
||||
**Worker Restart**: `npm run worker:restart`
|
||||
**Worker Logs**: `npm run worker:logs`
|
||||
**Version Bump**: `/skill version-bump`
|
||||
**Usage Analysis**: `npm run usage:today`
|
||||
**Viewer UI**: http://localhost:37777 (auto-starts with worker)
|
||||
|
||||
@@ -0,0 +1,614 @@
|
||||
# Implementation Plan: ROI Metrics & Discovery Cost Tracking
|
||||
|
||||
**Feature**: Display token discovery costs alongside observations to demonstrate knowledge reuse ROI
|
||||
**Branch**: `enhancement/roi`
|
||||
**Issue**: #104
|
||||
**Priority**: HIGH (needed for YC application amendment)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Capture token usage from Agent SDK, store as "discovery cost" with each observation, and display metrics in SessionStart context to prove that claude-mem reduces token consumption by 50-75% through knowledge reuse.
|
||||
|
||||
### The Value Proposition
|
||||
|
||||
**Session 1**: Claude spends 4,000 tokens discovering "how Stop hooks work"
|
||||
**Sessions 2-5**: Claude reads 163-token observation instead of re-discovering
|
||||
**Savings**: 15,348 tokens (77% reduction) over 5 sessions
|
||||
|
||||
This feature makes that ROI **visible and measurable** for both users and Claude.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
Agent SDK Messages (with usage)
|
||||
↓
|
||||
SDKAgent captures usage data
|
||||
↓
|
||||
ActiveSession tracks cumulative tokens
|
||||
↓
|
||||
Observations stored with discovery_tokens
|
||||
↓
|
||||
Context hook displays metrics
|
||||
↓
|
||||
User/Claude sees ROI
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Capture Token Usage from Agent SDK
|
||||
|
||||
**File**: `src/services/worker/SDKAgent.ts`
|
||||
|
||||
**Changes**:
|
||||
1. Extract usage data from assistant messages (lines 64-86)
|
||||
2. Track cumulative session tokens in ActiveSession
|
||||
3. Pass cumulative tokens when storing observations
|
||||
|
||||
**Code Changes**:
|
||||
|
||||
```typescript
|
||||
// Line ~70: After extracting textContent, add:
|
||||
const usage = message.message.usage;
|
||||
if (usage) {
|
||||
session.cumulativeInputTokens += usage.input_tokens || 0;
|
||||
session.cumulativeOutputTokens += usage.output_tokens || 0;
|
||||
|
||||
// Cache creation counts as discovery, cache read doesn't
|
||||
if (usage.cache_creation_input_tokens) {
|
||||
session.cumulativeInputTokens += usage.cache_creation_input_tokens;
|
||||
}
|
||||
|
||||
logger.debug('SDK', 'Token usage captured', {
|
||||
sessionId: session.sessionDbId,
|
||||
inputTokens: usage.input_tokens,
|
||||
outputTokens: usage.output_tokens,
|
||||
cumulativeInput: session.cumulativeInputTokens,
|
||||
cumulativeOutput: session.cumulativeOutputTokens
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// Line ~213-218: Pass discovery tokens when storing
|
||||
const { id: obsId, createdAtEpoch } = this.dbManager.getSessionStore().storeObservation(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
session.cumulativeInputTokens + session.cumulativeOutputTokens // Add discovery cost
|
||||
);
|
||||
```
|
||||
|
||||
**Edge Cases**:
|
||||
- Handle missing usage data (default to 0)
|
||||
- Cache tokens: `cache_creation_input_tokens` counts as discovery, `cache_read_input_tokens` doesn't
|
||||
- Multiple observations per response: Each gets snapshot of cumulative tokens at creation time
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Update ActiveSession Type
|
||||
|
||||
**File**: `src/services/worker-types.ts`
|
||||
|
||||
**Changes**: Add token tracking fields to ActiveSession interface
|
||||
|
||||
```typescript
|
||||
export interface ActiveSession {
|
||||
sessionDbId: number;
|
||||
sdkSessionId: string | null;
|
||||
claudeSessionId: string;
|
||||
project: string;
|
||||
userPrompt: string;
|
||||
lastPromptNumber: number;
|
||||
pendingMessages: PendingMessage[];
|
||||
abortController: AbortController;
|
||||
startTime: number;
|
||||
cumulativeInputTokens: number; // NEW: Track input tokens
|
||||
cumulativeOutputTokens: number; // NEW: Track output tokens
|
||||
}
|
||||
```
|
||||
|
||||
**Initialization**: When creating new session in SessionManager.initializeSession, set:
|
||||
```typescript
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Database Schema Migration
|
||||
|
||||
**File**: `src/services/sqlite/migrations.ts`
|
||||
|
||||
**Add Migration**: Create migration #8 (next available number)
|
||||
|
||||
```typescript
|
||||
{
|
||||
version: 8,
|
||||
name: 'add_discovery_tokens',
|
||||
up: (db: Database) => {
|
||||
// Add discovery_tokens to observations
|
||||
db.exec(`
|
||||
ALTER TABLE observations
|
||||
ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
|
||||
`);
|
||||
|
||||
// Add discovery_tokens to summaries
|
||||
db.exec(`
|
||||
ALTER TABLE summaries
|
||||
ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
|
||||
`);
|
||||
|
||||
logger.info('DB', 'Migration 8: Added discovery_tokens columns');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Why summaries too?**: Summaries represent accumulated session work, so they should also show total discovery cost.
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Update SessionStore
|
||||
|
||||
**File**: `src/services/sqlite/SessionStore.ts`
|
||||
|
||||
**Changes**:
|
||||
|
||||
1. Update `storeObservation` signature (around line ~1000):
|
||||
```typescript
|
||||
storeObservation(
|
||||
sessionId: string,
|
||||
project: string,
|
||||
observation: ParsedObservation,
|
||||
promptNumber: number,
|
||||
discoveryTokens: number = 0 // NEW parameter
|
||||
): { id: number; createdAtEpoch: number }
|
||||
```
|
||||
|
||||
2. Update INSERT statement to include discovery_tokens:
|
||||
```typescript
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO observations (
|
||||
session_id,
|
||||
project,
|
||||
type,
|
||||
title,
|
||||
subtitle,
|
||||
narrative,
|
||||
facts,
|
||||
concepts,
|
||||
files_read,
|
||||
files_modified,
|
||||
prompt_number,
|
||||
discovery_tokens, -- NEW
|
||||
created_at_epoch
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
sessionId,
|
||||
project,
|
||||
observation.type,
|
||||
observation.title,
|
||||
observation.subtitle || '',
|
||||
observation.narrative || '',
|
||||
JSON.stringify(observation.facts || []),
|
||||
JSON.stringify(observation.concepts || []),
|
||||
JSON.stringify(observation.files || []),
|
||||
JSON.stringify([]),
|
||||
promptNumber,
|
||||
discoveryTokens, // NEW
|
||||
createdAtEpoch
|
||||
);
|
||||
```
|
||||
|
||||
3. Update `storeSummary` similarly (around line ~1150):
|
||||
```typescript
|
||||
storeSummary(
|
||||
sessionId: string,
|
||||
project: string,
|
||||
summary: ParsedSummary,
|
||||
promptNumber: number,
|
||||
discoveryTokens: number = 0 // NEW parameter
|
||||
): { id: number; createdAtEpoch: number }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Update Database Types
|
||||
|
||||
**File**: `src/services/sqlite/types.ts`
|
||||
|
||||
**Changes**: Add discovery_tokens to DBObservation and DBSummary interfaces
|
||||
|
||||
```typescript
|
||||
export interface DBObservation {
|
||||
id: number;
|
||||
session_id: string;
|
||||
project: string;
|
||||
type: 'decision' | 'bugfix' | 'feature' | 'refactor' | 'discovery' | 'change';
|
||||
title: string;
|
||||
subtitle: string;
|
||||
narrative: string | null;
|
||||
facts: string; // JSON array
|
||||
concepts: string; // JSON array
|
||||
files_read: string; // JSON array
|
||||
files_modified: string; // JSON array
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // NEW
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
export interface DBSummary {
|
||||
id: number;
|
||||
session_id: string;
|
||||
request: string;
|
||||
investigated: string | null;
|
||||
learned: string | null;
|
||||
completed: string | null;
|
||||
next_steps: string | null;
|
||||
notes: string | null;
|
||||
project: string;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // NEW
|
||||
created_at_epoch: number;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Update Search Queries
|
||||
|
||||
**File**: `src/services/sqlite/SessionSearch.ts`
|
||||
|
||||
**Changes**: Ensure all SELECT queries include discovery_tokens
|
||||
|
||||
Example (around line ~50, searchObservations):
|
||||
```typescript
|
||||
SELECT
|
||||
o.id,
|
||||
o.session_id,
|
||||
o.project,
|
||||
o.type,
|
||||
o.title,
|
||||
o.subtitle,
|
||||
o.narrative,
|
||||
o.facts,
|
||||
o.concepts,
|
||||
o.files_read,
|
||||
o.files_modified,
|
||||
o.prompt_number,
|
||||
o.discovery_tokens, -- NEW
|
||||
o.created_at_epoch,
|
||||
...
|
||||
```
|
||||
|
||||
**Affected methods**:
|
||||
- `searchObservations`
|
||||
- `getRecentObservations`
|
||||
- `getObservationsByType`
|
||||
- `getObservationsByConcept`
|
||||
- `getObservationsByFile`
|
||||
- All other observation query methods
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Update Context Hook Display
|
||||
|
||||
**File**: `src/hooks/context-hook.ts`
|
||||
|
||||
**Changes**: Display discovery costs and ROI metrics in SessionStart context
|
||||
|
||||
**Section 1: Add Aggregate Metrics** (insert after line ~250, before observation table)
|
||||
|
||||
```typescript
|
||||
// Calculate aggregate metrics
|
||||
const totalObservations = observations.length;
|
||||
const totalReadTokens = observations.reduce((sum, obs) => {
|
||||
// Estimate read tokens from observation size
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
return sum + Math.ceil(obsSize / 4); // Rough token estimate
|
||||
}, 0);
|
||||
const totalDiscoveryTokens = observations.reduce((sum, obs) => sum + (obs.discovery_tokens || 0), 0);
|
||||
const savings = totalDiscoveryTokens - totalReadTokens;
|
||||
const savingsPercent = totalDiscoveryTokens > 0
|
||||
? Math.round((savings / totalDiscoveryTokens) * 100)
|
||||
: 0;
|
||||
|
||||
output += `\n📊 **Context Economics**:\n`;
|
||||
output += ` - Loading: ${totalObservations} observations (${totalReadTokens.toLocaleString()} tokens to read)\n`;
|
||||
output += ` - Discovery investment: ${totalDiscoveryTokens.toLocaleString()} tokens spent by previous sessions\n`;
|
||||
output += ` - Your savings: ${savings.toLocaleString()} tokens (${savingsPercent}% reduction from reuse)\n\n`;
|
||||
```
|
||||
|
||||
**Section 2: Update Table Format** (around line ~280)
|
||||
|
||||
```typescript
|
||||
output += `| ID | Time | T | Title | Read | Discovery |\n`;
|
||||
output += `|----|------|---|-------|------|-----------||\n`;
|
||||
|
||||
for (const obs of observations) {
|
||||
const time = formatTime(obs.created_at_epoch);
|
||||
const emoji = getTypeEmoji(obs.type);
|
||||
const title = truncate(obs.title, 50);
|
||||
|
||||
// Estimate read tokens (observation size in tokens)
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
const readTokens = Math.ceil(obsSize / 4);
|
||||
|
||||
const discoveryTokens = obs.discovery_tokens || 0;
|
||||
const discoveryDisplay = discoveryTokens > 0
|
||||
? `🔍 ${discoveryTokens.toLocaleString()}`
|
||||
: '-';
|
||||
|
||||
output += `| #${obs.id} | ${time} | ${emoji} | ${title} | ~${readTokens} | ${discoveryDisplay} |\n`;
|
||||
}
|
||||
```
|
||||
|
||||
**Section 3: Add Footer Explanation** (after table)
|
||||
|
||||
```typescript
|
||||
output += `\n💡 **Column Key**:\n`;
|
||||
output += ` - **Read**: Tokens to read this observation (cost to learn it now)\n`;
|
||||
output += ` - **Discovery**: Tokens Previous Claude spent exploring/researching this topic\n`;
|
||||
output += `\n**ROI**: Reading these learnings instead of re-discovering saves ${savingsPercent}% tokens\n`;
|
||||
```
|
||||
|
||||
**Edge Case**: Handle old observations without discovery_tokens (show '-' or 0)
|
||||
|
||||
---
|
||||
|
||||
### Phase 8: Update Chroma Sync (Optional)
|
||||
|
||||
**File**: `src/services/sync/ChromaSync.ts`
|
||||
|
||||
**Changes**: Include discovery_tokens in vector metadata
|
||||
|
||||
```typescript
|
||||
// Around line ~100, syncObservation metadata
|
||||
metadata: {
|
||||
session_id: sessionId,
|
||||
project: project,
|
||||
type: observation.type,
|
||||
title: observation.title,
|
||||
prompt_number: promptNumber,
|
||||
discovery_tokens: discoveryTokens, // NEW
|
||||
created_at_epoch: createdAtEpoch,
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
**Why?**: Enables semantic search to factor in discovery cost for relevance scoring (future enhancement)
|
||||
|
||||
---
|
||||
|
||||
## Testing Plan
|
||||
|
||||
### Unit Tests
|
||||
|
||||
1. **Token Capture Test**:
|
||||
- Mock Agent SDK response with usage data
|
||||
- Verify ActiveSession.cumulativeTokens increments correctly
|
||||
- Test cache token handling (creation counts, read doesn't)
|
||||
|
||||
2. **Storage Test**:
|
||||
- Create observation with discovery_tokens
|
||||
- Verify database stores correctly
|
||||
- Query back and verify field present
|
||||
|
||||
3. **Display Test**:
|
||||
- Create test observations with varying discovery costs
|
||||
- Run context-hook
|
||||
- Verify metrics calculate correctly
|
||||
- Verify table displays both Read and Discovery columns
|
||||
|
||||
### Integration Tests
|
||||
|
||||
1. **Full Session Flow**:
|
||||
- Start new session
|
||||
- Trigger multiple tool executions
|
||||
- Generate observations
|
||||
- Verify cumulative tokens accumulate
|
||||
- Check context displays metrics
|
||||
|
||||
2. **Migration Test**:
|
||||
- Backup existing database
|
||||
- Run migration #8
|
||||
- Verify columns added
|
||||
- Verify existing data intact (discovery_tokens = 0)
|
||||
- Test new observations store correctly
|
||||
|
||||
### Manual Testing
|
||||
|
||||
1. **Real Usage Scenario**:
|
||||
- Start fresh Claude Code session
|
||||
- Perform research task (read files, search codebase)
|
||||
- Generate observations via claude-mem
|
||||
- Check database for discovery_tokens values
|
||||
- Start new session, verify context shows metrics
|
||||
|
||||
2. **YC Demo Data**:
|
||||
- Run 5 sessions on same topic
|
||||
- Collect token data for each session
|
||||
- Calculate actual ROI (Session 1 cost vs Sessions 2-5)
|
||||
- Screenshot metrics for YC application
|
||||
|
||||
---
|
||||
|
||||
## Rollout Plan
|
||||
|
||||
### Phase 1: Data Collection (Week 1)
|
||||
- Deploy migration and token capture
|
||||
- Run without displaying metrics yet
|
||||
- Verify data quality and accuracy
|
||||
- Fix any issues with token tracking
|
||||
|
||||
### Phase 2: Display Metrics (Week 2)
|
||||
- Enable context hook display
|
||||
- Gather user feedback
|
||||
- Iterate on presentation format
|
||||
- Document any edge cases
|
||||
|
||||
### Phase 3: YC Application (Week 2-3)
|
||||
- Collect empirical data from real usage
|
||||
- Generate charts/graphs showing ROI
|
||||
- Write case study with actual numbers
|
||||
- Amend YC application with proof
|
||||
|
||||
### Phase 4: Public Launch (Week 4)
|
||||
- Blog post explaining the feature
|
||||
- Update README with ROI metrics
|
||||
- Submit to HN/Reddit with data
|
||||
- Reach out to Anthropic with findings
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
**Technical Success**:
|
||||
- ✅ Token capture accuracy: >95% of SDK responses captured
|
||||
- ✅ Database migration: 0 data loss, all observations migrated
|
||||
- ✅ Display accuracy: Metrics match raw data within 5%
|
||||
|
||||
**Business Success**:
|
||||
- ✅ Demonstrate 50-75% token reduction across 10+ sessions
|
||||
- ✅ YC application strengthened with empirical data
|
||||
- ✅ User/Claude understanding of ROI improves (survey/feedback)
|
||||
|
||||
**Strategic Success**:
|
||||
- ✅ Proof that memory optimization reduces infrastructure needs
|
||||
- ✅ Data compelling enough for Anthropic partnership discussion
|
||||
- ✅ Foundation for enterprise licensing ROI calculator
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Token Attribution**:
|
||||
- Should each observation get cumulative session tokens, or split proportionally?
|
||||
- **Decision**: Use cumulative (simpler, shows total cost at that point)
|
||||
|
||||
2. **Cache Tokens**:
|
||||
- How to handle cache_read_input_tokens in ROI calculation?
|
||||
- **Decision**: Don't count cache reads as discovery (they're already discovered)
|
||||
|
||||
3. **Display Format**:
|
||||
- Show raw token counts or human-readable format (K, M)?
|
||||
- **Decision**: Use toLocaleString() for readability (e.g., "4,000" not "4K")
|
||||
|
||||
4. **Pricing Display**:
|
||||
- Should we show dollar costs too, or just tokens?
|
||||
- **Decision**: Tokens only initially. Pricing varies by model/plan, adds complexity
|
||||
|
||||
5. **Historical Data**:
|
||||
- What to do with old observations without discovery_tokens?
|
||||
- **Decision**: Show as 0 or '-', document limitation
|
||||
|
||||
---
|
||||
|
||||
## Files Modified Summary
|
||||
|
||||
**Core Implementation**:
|
||||
- `src/services/worker/SDKAgent.ts` - Capture usage, pass to storage
|
||||
- `src/services/worker-types.ts` - Add cumulative token fields
|
||||
- `src/services/sqlite/migrations.ts` - Migration #8 for discovery_tokens
|
||||
- `src/services/sqlite/SessionStore.ts` - Store discovery tokens
|
||||
- `src/services/sqlite/types.ts` - Update interfaces
|
||||
- `src/services/sqlite/SessionSearch.ts` - Include in queries
|
||||
- `src/hooks/context-hook.ts` - Display metrics
|
||||
|
||||
**Optional**:
|
||||
- `src/services/sync/ChromaSync.ts` - Include in vector metadata
|
||||
- `src/services/worker/SessionManager.ts` - Initialize cumulative tokens
|
||||
|
||||
**Documentation**:
|
||||
- `CLAUDE.md` - Update with new feature
|
||||
- `README.md` - Add ROI metrics section
|
||||
- Issue #104 - Track implementation progress
|
||||
|
||||
---
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
**Day 1** (Tomorrow):
|
||||
- [ ] Create branch ✅
|
||||
- [ ] Write implementation plan ✅
|
||||
- [ ] Phase 1: Capture token usage (2 hours)
|
||||
- [ ] Phase 2: Update types (30 min)
|
||||
- [ ] Phase 3: Database migration (1 hour)
|
||||
|
||||
**Day 2**:
|
||||
- [ ] Phase 4: Update SessionStore (1 hour)
|
||||
- [ ] Phase 5: Update types (30 min)
|
||||
- [ ] Phase 6: Update search queries (1 hour)
|
||||
- [ ] Testing: Unit tests (2 hours)
|
||||
|
||||
**Day 3**:
|
||||
- [ ] Phase 7: Update context hook display (2 hours)
|
||||
- [ ] Testing: Integration tests (2 hours)
|
||||
- [ ] Manual testing and iteration (2 hours)
|
||||
|
||||
**Day 4**:
|
||||
- [ ] Collect real usage data (ongoing throughout day)
|
||||
- [ ] Generate YC metrics/charts (2 hours)
|
||||
- [ ] Amend YC application (2 hours)
|
||||
- [ ] Documentation updates (1 hour)
|
||||
|
||||
**Total**: ~20 hours of development over 4 days
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
**Risk 1**: Agent SDK usage data incomplete or missing
|
||||
**Mitigation**: Default to 0, log warnings, don't break existing functionality
|
||||
|
||||
**Risk 2**: Migration fails on large databases
|
||||
**Mitigation**: Test on database copy first, add rollback mechanism
|
||||
|
||||
**Risk 3**: Token estimates inaccurate
|
||||
**Mitigation**: Document methodology, provide "rough estimate" disclaimer
|
||||
|
||||
**Risk 4**: Display too noisy/overwhelming
|
||||
**Mitigation**: Make display configurable via settings, start collapsed
|
||||
|
||||
**Risk 5**: YC data not compelling enough
|
||||
**Mitigation**: Run on diverse projects, cherry-pick best examples, be honest about limitations
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Create branch `enhancement/roi`
|
||||
2. ✅ Write implementation plan
|
||||
3. Start Phase 1: Implement token capture in SDKAgent.ts
|
||||
4. Run manual test to verify usage data captured
|
||||
5. Continue through phases sequentially
|
||||
6. Collect data for YC application by end of week
|
||||
|
||||
---
|
||||
|
||||
## Notes for Tomorrow
|
||||
|
||||
**Start here**: `src/services/worker/SDKAgent.ts` line 64-86
|
||||
**Key insight**: `message.message.usage` contains the token data
|
||||
**Don't forget**: Initialize cumulative tokens to 0 in SessionManager
|
||||
**Test with**: Simple session that reads a few files and creates 1-2 observations
|
||||
|
||||
**The goal**: By end of week, have real numbers showing 50-75% token savings to prove the hypothesis and strengthen YC application.
|
||||
|
||||
---
|
||||
|
||||
*This plan represents ~20 hours of focused development. Prioritize getting Phase 1-7 working correctly over perfection. The YC data is the critical deliverable.*
|
||||
@@ -1,468 +0,0 @@
|
||||
# Plan: Display Complete Observation Data in Viewer UI
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### What's Currently Shown (5 fields)
|
||||
- ✅ **type** - Displayed as chip/badge (e.g., "discovery", "bugfix")
|
||||
- ✅ **project** - Shown in card header
|
||||
- ✅ **title** - Main card title (shows "Untitled" if null)
|
||||
- ✅ **subtitle** - Optional subheading
|
||||
- ✅ **id + created_at** - Metadata line (e.g., "#1 • 2 hours ago")
|
||||
|
||||
### What's Hidden (10+ fields)
|
||||
- ❌ **narrative** - Detailed explanation text (MOST IMPORTANT)
|
||||
- ❌ **facts** - JSON array of key facts (structured bullet points)
|
||||
- ❌ **concepts** - JSON array of concept tags (e.g., "problem-solution", "gotcha")
|
||||
- ❌ **files_read** - JSON array of file paths that were read
|
||||
- ❌ **files_modified** - JSON array of file paths that were modified
|
||||
- ❌ **text** - Legacy unstructured text field (deprecated but still populated)
|
||||
- ❌ **prompt_number** - Which user prompt triggered this observation
|
||||
- ❌ **sdk_session_id** - Session identifier
|
||||
|
||||
### Database Schema (Actual Structure)
|
||||
|
||||
```sql
|
||||
observations table:
|
||||
- id (INTEGER PRIMARY KEY)
|
||||
- sdk_session_id (TEXT)
|
||||
- project (TEXT)
|
||||
- type (TEXT: decision, bugfix, feature, refactor, discovery, change)
|
||||
- created_at (TEXT ISO timestamp)
|
||||
- created_at_epoch (INTEGER milliseconds)
|
||||
- prompt_number (INTEGER nullable)
|
||||
- title (TEXT nullable)
|
||||
- subtitle (TEXT nullable)
|
||||
- narrative (TEXT nullable) -- Rich detailed explanation
|
||||
- text (TEXT nullable) -- Legacy field
|
||||
- facts (TEXT nullable) -- JSON array of key facts
|
||||
- concepts (TEXT nullable) -- JSON array of concept tags
|
||||
- files_read (TEXT nullable) -- JSON array of file paths
|
||||
- files_modified (TEXT nullable) -- JSON array of file paths
|
||||
```
|
||||
|
||||
### Issues Found
|
||||
|
||||
1. **Type Definition Mismatch**: Three different type definitions exist:
|
||||
- Actual database schema (most complete)
|
||||
- `worker-types.ts` Observation interface (flattened, has wrong field names)
|
||||
- `viewer/types.ts` Observation interface (minimal subset)
|
||||
|
||||
2. **Data Loss**: Rich fields are stored in DB but not transmitted to UI:
|
||||
- narrative, facts, files_read, files_modified all missing from API
|
||||
|
||||
3. **PaginationHelper Query Bug**: Selects non-existent fields:
|
||||
- `session_db_id` (should be `sdk_session_id`)
|
||||
- `claude_session_id` (doesn't exist in observations table)
|
||||
- `files` (should be `files_read` + `files_modified`)
|
||||
|
||||
## Proposed Implementation Plan
|
||||
|
||||
### Phase 1: Fix Data Layer
|
||||
|
||||
#### 1.1 Update Viewer Type Definitions
|
||||
**File**: `src/ui/viewer/types.ts`
|
||||
|
||||
```typescript
|
||||
export interface Observation {
|
||||
id: number;
|
||||
sdk_session_id: string;
|
||||
project: string;
|
||||
type: string;
|
||||
title: string | null;
|
||||
subtitle: string | null;
|
||||
narrative: string | null; // NEW - detailed explanation
|
||||
text: string | null; // Legacy field
|
||||
facts: string | null; // NEW - JSON array of key facts
|
||||
concepts: string | null; // NEW - JSON array of concept tags
|
||||
files_read: string | null; // NEW - JSON array of file paths
|
||||
files_modified: string | null; // NEW - JSON array of file paths
|
||||
prompt_number: number | null; // NEW - which prompt triggered this
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
```
|
||||
|
||||
#### 1.2 Fix PaginationHelper SQL Query
|
||||
**File**: `src/services/worker/PaginationHelper.ts` (around line 26)
|
||||
|
||||
**Current (BROKEN)**:
|
||||
```typescript
|
||||
const fields = 'id, session_db_id, claude_session_id, project, type, title, subtitle, text, concepts, files, prompt_number, created_at, created_at_epoch';
|
||||
```
|
||||
|
||||
**Fixed**:
|
||||
```typescript
|
||||
const fields = 'id, sdk_session_id, project, type, title, subtitle, narrative, text, facts, concepts, files_read, files_modified, prompt_number, created_at, created_at_epoch';
|
||||
```
|
||||
|
||||
#### 1.3 Update Worker Service v2 Response Mapping
|
||||
**File**: `src/services/worker-service-v2.ts`
|
||||
|
||||
Ensure the `/api/observations` endpoint properly maps all fields from database to response. May need to parse JSON fields (facts, concepts, files_read, files_modified) if they're stored as JSON strings.
|
||||
|
||||
### Phase 2: Redesign UI Component
|
||||
|
||||
#### 2.1 Update ObservationCard Component
|
||||
**File**: `src/ui/viewer/components/ObservationCard.tsx`
|
||||
|
||||
**New Structure**:
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ [type badge] [project] │ ← Header (always visible)
|
||||
├─────────────────────────────────────────┤
|
||||
│ Title │ ← Always visible
|
||||
│ Subtitle (if present) │ ← Always visible
|
||||
│ #123 • 2 hours ago [▼ More]│ ← Metadata + Expand button
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─ EXPANDED CONTENT (when opened) ───┐ │
|
||||
│ │ │ │
|
||||
│ │ 📝 Narrative │ │
|
||||
│ │ ─────────────────────────────────── │ │
|
||||
│ │ Detailed explanation text... │ │
|
||||
│ │ │ │
|
||||
│ │ 📌 Key Facts │ │
|
||||
│ │ ─────────────────────────────────── │ │
|
||||
│ │ • Fact 1 │ │
|
||||
│ │ • Fact 2 │ │
|
||||
│ │ • Fact 3 │ │
|
||||
│ │ │ │
|
||||
│ │ 🏷️ Concepts │ │
|
||||
│ │ ─────────────────────────────────── │ │
|
||||
│ │ [problem-solution] [discovery] │ │
|
||||
│ │ │ │
|
||||
│ │ 📁 Files │ │
|
||||
│ │ ─────────────────────────────────── │ │
|
||||
│ │ 📖 Read: │ │
|
||||
│ │ src/hooks/save-hook.ts │ │
|
||||
│ │ src/services/worker.ts │ │
|
||||
│ │ ✏️ Modified: │ │
|
||||
│ │ src/hooks/save-hook.ts │ │
|
||||
│ │ │ │
|
||||
│ │ 🔗 Session Info │ │
|
||||
│ │ ─────────────────────────────────── │ │
|
||||
│ │ Prompt #5 • Session: abc123... │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Component Logic**:
|
||||
```typescript
|
||||
const ObservationCard = ({ observation }) => {
|
||||
const [isExpanded, setIsExpanded] = useState(false);
|
||||
|
||||
// Parse JSON fields
|
||||
const facts = observation.facts ? JSON.parse(observation.facts) : [];
|
||||
const concepts = observation.concepts ? JSON.parse(observation.concepts) : [];
|
||||
const filesRead = observation.files_read ? JSON.parse(observation.files_read) : [];
|
||||
const filesModified = observation.files_modified ? JSON.parse(observation.files_modified) : [];
|
||||
|
||||
return (
|
||||
<div className={`card ${isExpanded ? 'card-expanded' : ''}`}>
|
||||
{/* Header - always visible */}
|
||||
<div className="card-header">
|
||||
<span className={`card-type type-${observation.type}`}>
|
||||
{observation.type}
|
||||
</span>
|
||||
<span className="card-project">{observation.project}</span>
|
||||
</div>
|
||||
|
||||
{/* Title/Subtitle - always visible */}
|
||||
<div className="card-title">{observation.title || 'Untitled'}</div>
|
||||
{observation.subtitle && (
|
||||
<div className="card-subtitle">{observation.subtitle}</div>
|
||||
)}
|
||||
|
||||
{/* Metadata + Expand button - always visible */}
|
||||
<div className="card-meta">
|
||||
<span>#{observation.id} • {formatDate(observation.created_at_epoch)}</span>
|
||||
<button
|
||||
className="expand-toggle"
|
||||
onClick={() => setIsExpanded(!isExpanded)}
|
||||
>
|
||||
{isExpanded ? '▲ Less' : '▼ More'}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Expanded content - conditional */}
|
||||
{isExpanded && (
|
||||
<div className="card-expanded-content">
|
||||
|
||||
{/* Narrative Section */}
|
||||
{observation.narrative && (
|
||||
<div className="card-section">
|
||||
<div className="section-header">📝 Narrative</div>
|
||||
<div className="section-content narrative">
|
||||
{observation.narrative}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Facts Section */}
|
||||
{facts.length > 0 && (
|
||||
<div className="card-section">
|
||||
<div className="section-header">📌 Key Facts</div>
|
||||
<ul className="section-content facts-list">
|
||||
{facts.map((fact, i) => (
|
||||
<li key={i}>{fact}</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Concepts Section */}
|
||||
{concepts.length > 0 && (
|
||||
<div className="card-section">
|
||||
<div className="section-header">🏷️ Concepts</div>
|
||||
<div className="section-content concepts">
|
||||
{concepts.map((concept, i) => (
|
||||
<span key={i} className="concept-tag">{concept}</span>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Files Section */}
|
||||
{(filesRead.length > 0 || filesModified.length > 0) && (
|
||||
<div className="card-section">
|
||||
<div className="section-header">📁 Files</div>
|
||||
<div className="section-content files">
|
||||
{filesRead.length > 0 && (
|
||||
<div className="file-group">
|
||||
<div className="file-group-label">📖 Read:</div>
|
||||
{filesRead.map((file, i) => (
|
||||
<div key={i} className="file-path">{file}</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
{filesModified.length > 0 && (
|
||||
<div className="file-group">
|
||||
<div className="file-group-label">✏️ Modified:</div>
|
||||
{filesModified.map((file, i) => (
|
||||
<div key={i} className="file-path">{file}</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Session Info Section */}
|
||||
<div className="card-section">
|
||||
<div className="section-header">🔗 Session Info</div>
|
||||
<div className="section-content session-info">
|
||||
{observation.prompt_number && (
|
||||
<span>Prompt #{observation.prompt_number}</span>
|
||||
)}
|
||||
{observation.sdk_session_id && (
|
||||
<span className="session-id">
|
||||
Session: {observation.sdk_session_id.substring(0, 8)}...
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### Phase 3: Style Enhancements
|
||||
|
||||
#### 3.1 Update Styles
|
||||
**File**: `src/ui/viewer/styles.css`
|
||||
|
||||
**New CSS Classes Needed**:
|
||||
```css
|
||||
/* Expanded card state */
|
||||
.card-expanded {
|
||||
/* Maybe increase shadow or border when expanded */
|
||||
}
|
||||
|
||||
/* Expand toggle button */
|
||||
.expand-toggle {
|
||||
background: none;
|
||||
border: none;
|
||||
color: var(--text-secondary);
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
padding: 4px 8px;
|
||||
border-radius: 4px;
|
||||
}
|
||||
.expand-toggle:hover {
|
||||
background: var(--bg-secondary);
|
||||
}
|
||||
|
||||
/* Expanded content container */
|
||||
.card-expanded-content {
|
||||
margin-top: 16px;
|
||||
padding-top: 16px;
|
||||
border-top: 1px solid var(--border-color);
|
||||
animation: expandDown 0.2s ease-out;
|
||||
}
|
||||
|
||||
@keyframes expandDown {
|
||||
from {
|
||||
opacity: 0;
|
||||
transform: translateY(-8px);
|
||||
}
|
||||
to {
|
||||
opacity: 1;
|
||||
transform: translateY(0);
|
||||
}
|
||||
}
|
||||
|
||||
/* Section styling */
|
||||
.card-section {
|
||||
margin-bottom: 16px;
|
||||
}
|
||||
.card-section:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
|
||||
.section-header {
|
||||
font-weight: 600;
|
||||
font-size: 13px;
|
||||
color: var(--text-primary);
|
||||
margin-bottom: 8px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.section-content {
|
||||
padding-left: 20px;
|
||||
color: var(--text-secondary);
|
||||
font-size: 13px;
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
/* Narrative styling */
|
||||
.narrative {
|
||||
max-height: 300px;
|
||||
overflow-y: auto;
|
||||
white-space: pre-wrap;
|
||||
word-wrap: break-word;
|
||||
}
|
||||
|
||||
/* Facts list styling */
|
||||
.facts-list {
|
||||
list-style: disc;
|
||||
margin: 0;
|
||||
padding-left: 20px;
|
||||
}
|
||||
.facts-list li {
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
|
||||
/* Concepts tags */
|
||||
.concepts {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 6px;
|
||||
}
|
||||
.concept-tag {
|
||||
background: var(--accent-bg);
|
||||
color: var(--accent-text);
|
||||
padding: 4px 10px;
|
||||
border-radius: 12px;
|
||||
font-size: 11px;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
/* File paths */
|
||||
.file-group {
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.file-group:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
.file-group-label {
|
||||
font-weight: 500;
|
||||
margin-bottom: 4px;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
.file-path {
|
||||
font-family: 'SF Mono', 'Monaco', 'Courier New', monospace;
|
||||
font-size: 12px;
|
||||
padding: 4px 8px;
|
||||
background: var(--code-bg);
|
||||
border-radius: 4px;
|
||||
margin-bottom: 2px;
|
||||
overflow-x: auto;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
/* Session info */
|
||||
.session-info {
|
||||
display: flex;
|
||||
gap: 16px;
|
||||
font-size: 12px;
|
||||
}
|
||||
.session-id {
|
||||
font-family: 'SF Mono', 'Monaco', 'Courier New', monospace;
|
||||
color: var(--text-tertiary);
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation Steps (In Order)
|
||||
|
||||
1. **Fix PaginationHelper query** (src/services/worker/PaginationHelper.ts)
|
||||
- Update SQL SELECT to use correct field names
|
||||
- Test with `npm run worker:restart:v2`
|
||||
|
||||
2. **Update viewer type definitions** (src/ui/viewer/types.ts)
|
||||
- Add all missing fields to Observation interface
|
||||
|
||||
3. **Verify worker service v2 mapping** (src/services/worker-service-v2.ts)
|
||||
- Ensure `/api/observations` returns all fields
|
||||
- Test API response with curl or browser
|
||||
|
||||
4. **Update ObservationCard component** (src/ui/viewer/components/ObservationCard.tsx)
|
||||
- Add expand/collapse state
|
||||
- Add all new sections (narrative, facts, concepts, files, session)
|
||||
- Add expand toggle button
|
||||
|
||||
5. **Update styles** (src/ui/viewer/styles.css)
|
||||
- Add all new CSS classes for expanded content
|
||||
- Add animations for smooth expand/collapse
|
||||
- Style sections, lists, tags, file paths
|
||||
|
||||
6. **Build and test**
|
||||
```bash
|
||||
npm run build
|
||||
npm run sync-marketplace
|
||||
npm run worker:restart:v2
|
||||
```
|
||||
|
||||
7. **Manual testing**
|
||||
- Open http://localhost:37777
|
||||
- Click expand button on observations
|
||||
- Verify all fields display correctly
|
||||
- Test light/dark mode
|
||||
- Test with observations that have missing fields (graceful fallback)
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] All database fields are fetched in API query
|
||||
- [ ] All fields are properly typed in TypeScript interfaces
|
||||
- [ ] ObservationCard shows all data in expanded view
|
||||
- [ ] Expand/collapse animations work smoothly
|
||||
- [ ] File paths are formatted in monospace font
|
||||
- [ ] Concepts display as tag pills
|
||||
- [ ] Facts display as bulleted list
|
||||
- [ ] Narrative text wraps properly with scroll for long content
|
||||
- [ ] No console errors
|
||||
- [ ] Works in both light and dark themes
|
||||
|
||||
## Optional Enhancements (Future)
|
||||
|
||||
- [ ] Remember expanded state in localStorage (persist across page refresh)
|
||||
- [ ] Keyboard shortcuts (Space to expand/collapse focused card)
|
||||
- [ ] Click file paths to copy to clipboard
|
||||
- [ ] Search/filter by concepts or files
|
||||
- [ ] Syntax highlighting for code in narrative
|
||||
- [ ] Link session_id to session detail view
|
||||
@@ -17,7 +17,7 @@
|
||||
<img src="https://img.shields.io/badge/License-AGPL%203.0-blue.svg" alt="License">
|
||||
</a>
|
||||
<a href="package.json">
|
||||
<img src="https://img.shields.io/badge/version-5.4.0-green.svg" alt="Version">
|
||||
<img src="https://img.shields.io/badge/version-5.5.1-green.svg" alt="Version">
|
||||
</a>
|
||||
<a href="package.json">
|
||||
<img src="https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg" alt="Node">
|
||||
@@ -69,7 +69,7 @@ Restart Claude Code. Context from previous sessions will automatically appear in
|
||||
|
||||
- 🧠 **Persistent Memory** - Context survives across sessions
|
||||
- 📊 **Progressive Disclosure** - Layered memory retrieval with token cost visibility
|
||||
- 🔍 **Skill-Based Search** - Query your project history with natural language (~2,250 token savings)
|
||||
- 🔍 **Skill-Based Search** - Query your project history with mem-search skill (~2,250 token savings)
|
||||
- 🖥️ **Web Viewer UI** - Real-time memory stream at http://localhost:37777
|
||||
- 🤖 **Automatic Operation** - No manual intervention required
|
||||
- 🔗 **Citations** - Reference past decisions with `claude-mem://` URIs
|
||||
@@ -143,23 +143,24 @@ npx mintlify dev
|
||||
|
||||
**Core Components:**
|
||||
|
||||
1. **7 Lifecycle Hook Scripts** - smart-install, context-hook, user-message-hook, new-hook, save-hook, summary-hook, cleanup-hook
|
||||
2. **Worker Service** - HTTP API on port 37777 with web viewer UI and 10 search endpoints, managed by PM2
|
||||
3. **SQLite Database** - Stores sessions, observations, summaries with FTS5 full-text search
|
||||
4. **Search Skill** - Natural language queries with progressive disclosure (~2,250 token savings vs MCP)
|
||||
5. **Chroma Vector Database** - Hybrid semantic + keyword search for intelligent context retrieval
|
||||
1. **6 Lifecycle Hooks** - context-hook, user-message-hook, new-hook, save-hook, summary-hook, cleanup-hook
|
||||
2. **Smart Install** - Cached dependency checker (pre-hook script, not a lifecycle hook)
|
||||
3. **Worker Service** - HTTP API on port 37777 with web viewer UI and 10 search endpoints, managed by PM2
|
||||
4. **SQLite Database** - Stores sessions, observations, summaries with FTS5 full-text search
|
||||
5. **mem-search Skill** - Natural language queries with progressive disclosure (~2,250 token savings vs MCP)
|
||||
6. **Chroma Vector Database** - Hybrid semantic + keyword search for intelligent context retrieval
|
||||
|
||||
See [Architecture Overview](docs/architecture/overview.mdx) for details.
|
||||
|
||||
---
|
||||
|
||||
## Skill-Based Search
|
||||
## mem-search Skill
|
||||
|
||||
Claude-Mem provides intelligent search through a skill that auto-invokes when you ask about past work:
|
||||
Claude-Mem provides intelligent search through the mem-search skill that auto-invokes when you ask about past work:
|
||||
|
||||
**How It Works:**
|
||||
- Just ask naturally: *"What did we do last session?"* or *"Did we fix this bug before?"*
|
||||
- Claude automatically invokes the search skill to find relevant context
|
||||
- Claude automatically invokes the mem-search skill to find relevant context
|
||||
- ~2,250 token savings per session start vs MCP approach
|
||||
|
||||
**Available Search Operations:**
|
||||
@@ -189,13 +190,21 @@ See [Search Tools Guide](docs/usage/search-tools.mdx) for detailed examples.
|
||||
|
||||
---
|
||||
|
||||
## What's New in v5.4.0
|
||||
## What's New in v5.5.1
|
||||
|
||||
**🎯 mem-search Skill Enhancement (v5.5.0):**
|
||||
|
||||
- **Improved Effectiveness**: Skill success rate increased from 67% to 100%
|
||||
- **Better Scope Differentiation**: Clear distinction from native conversation memory
|
||||
- **Enhanced Triggers**: Concrete triggers increased from 44% to 85%
|
||||
- **System-Specific Naming**: "mem-search" replaces generic "search" for clarity
|
||||
- **Comprehensive Documentation**: 17 total files with detailed operation guides
|
||||
|
||||
**🔍 Skill-Based Search Architecture (v5.4.0):**
|
||||
|
||||
- **Token Savings**: ~2,250 tokens per session start
|
||||
- **Progressive Disclosure**: Skill frontmatter (~250 tokens) vs MCP tool definitions (~2,500 tokens)
|
||||
- **Natural Language**: Just ask about past work - Claude auto-invokes the search skill
|
||||
- **Natural Language**: Just ask about past work - Claude auto-invokes the mem-search skill
|
||||
- **10 HTTP API Endpoints**: Fast, efficient search operations
|
||||
- **No User Action Required**: Migration is transparent
|
||||
|
||||
@@ -317,11 +326,7 @@ See [Development Guide](docs/development.mdx) for detailed instructions.
|
||||
|
||||
**Quick Diagnostic:**
|
||||
|
||||
Run the troubleshooting skill for automated diagnosis and fixes:
|
||||
|
||||
```
|
||||
/skill troubleshoot
|
||||
```
|
||||
If you're experiencing issues, describe the problem to Claude and the troubleshoot skill will automatically activate to diagnose and provide fixes.
|
||||
|
||||
**Common Issues:**
|
||||
|
||||
|
||||
@@ -1,129 +0,0 @@
|
||||
Plan: Migrate to Skill-Based Search (Deprecate MCP)
|
||||
|
||||
Goal
|
||||
|
||||
Replace MCP search tools with a skill-based approach, reducing session
|
||||
start context from ~2,500 tokens to ~250 tokens. Clean migration, no
|
||||
toggles.
|
||||
|
||||
Implementation Steps
|
||||
|
||||
1. Add HTTP API Endpoints to Worker Service
|
||||
|
||||
File: src/services/worker-service.ts
|
||||
|
||||
Add 10 new routes that wrap existing SessionSearch methods:
|
||||
- GET /api/search/observations?query=...&format=index&limit=20&project=...
|
||||
- GET /api/search/sessions?query=...&format=index&limit=20
|
||||
- GET /api/search/prompts?query=...&format=index&limit=20
|
||||
- GET /api/search/by-concept?concept=discovery&format=index&limit=5
|
||||
- GET /api/search/by-file?filePath=...&format=index&limit=10
|
||||
- GET /api/search/by-type?type=bugfix&format=index&limit=10
|
||||
- GET /api/context/recent?project=...&limit=3
|
||||
- GET /api/context/timeline?anchor=123&depth_before=10&depth_after=10
|
||||
- GET
|
||||
/api/timeline/by-query?query=...&mode=auto&depth_before=10&depth_after=10
|
||||
- GET /api/search/help - Returns available endpoints and usage docs
|
||||
|
||||
All endpoints return JSON. Skill parses and formats for readability.
|
||||
|
||||
2. Create Search Skill
|
||||
|
||||
File: plugin/skills/search/SKILL.md
|
||||
|
||||
Frontmatter:
|
||||
---
|
||||
name: search
|
||||
description: Search claude-mem persistent memory for past sessions,
|
||||
observations, bugs fixed, features implemented, decisions made, code
|
||||
changes, and previous work. Use when answering questions about history,
|
||||
finding past decisions, or researching previous implementations.
|
||||
---
|
||||
|
||||
Content: Instructions for all 9 search types using curl to call HTTP
|
||||
endpoints, formatting guidelines, common workflows.
|
||||
|
||||
3. Remove MCP Search Server
|
||||
|
||||
Files to modify:
|
||||
- Remove plugin/.mcp.json entry for claude-mem-search
|
||||
- Keep src/servers/search-server.ts for reference but don't build it
|
||||
- Update scripts/build-plugin.js to skip building search-server.mjs
|
||||
- Archive search-server implementation (don't delete, for reference)
|
||||
|
||||
4. Update Documentation
|
||||
|
||||
File: CLAUDE.md
|
||||
|
||||
Remove MCP search references, add skill search explanation:
|
||||
- Token savings: ~2,250 tokens per session
|
||||
- How skill auto-invokes (model-driven, not user-driven)
|
||||
- Available search operations
|
||||
- Examples of triggering searches
|
||||
|
||||
5. Add Migration Notice
|
||||
|
||||
File: CHANGELOG.md or release notes
|
||||
|
||||
Document the breaking change:
|
||||
## v5.4.0 - Skill-Based Search Migration
|
||||
|
||||
**BREAKING CHANGE**: MCP search tools have been replaced with a
|
||||
skill-based approach.
|
||||
|
||||
**What changed**:
|
||||
- Removed 9 MCP search tools (search_observations, search_sessions, etc.)
|
||||
- Added `search` skill that provides the same functionality
|
||||
- Reduced session start context by ~2,250 tokens
|
||||
|
||||
**Migration**: None required. Claude automatically uses the search skill
|
||||
when needed.
|
||||
The skill provides the same search capabilities with better token
|
||||
efficiency.
|
||||
|
||||
**Why**: Skill-based search uses progressive disclosure (~250 tokens for
|
||||
frontmatter)
|
||||
instead of loading all 9 tool definitions (~2,500 tokens) on every session
|
||||
start.
|
||||
|
||||
6. Testing Checklist
|
||||
|
||||
- All 10 HTTP endpoints return correct data
|
||||
- Skill auto-invokes when asking about past work
|
||||
- Skill successfully calls endpoints via curl
|
||||
- Skill formats results as readable markdown
|
||||
- Worker restart updates endpoints
|
||||
- Skill distributed correctly with plugin
|
||||
- No MCP search server registered
|
||||
- Session start context reduced by ~2,250 tokens
|
||||
|
||||
Token Impact
|
||||
|
||||
- Before: ~2,500 tokens (9 MCP tool definitions)
|
||||
- After: ~250 tokens (skill frontmatter only)
|
||||
- Savings: ~2,250 tokens per session start
|
||||
|
||||
User Experience
|
||||
|
||||
New behavior:
|
||||
- User: "What bug did we fix last session?"
|
||||
- Claude sees skill description matches → invokes search skill
|
||||
- Skill loads full instructions → uses curl to call HTTP API → formats
|
||||
results
|
||||
- User sees formatted answer
|
||||
|
||||
No user action required: Migration is transparent, searches work
|
||||
automatically.
|
||||
|
||||
Build & Deploy
|
||||
|
||||
npm run build # Builds skill, skips MCP server
|
||||
npm run sync-marketplace # Syncs plugin with skill
|
||||
npm run worker:restart # Restart worker with new HTTP endpoints
|
||||
|
||||
Rollout
|
||||
|
||||
1. Ship as breaking change in v5.4.0
|
||||
2. Update plugin marketplace listing
|
||||
3. All users get automatic token savings on update
|
||||
4. Archive MCP search implementation for reference
|
||||
@@ -1,73 +0,0 @@
|
||||
# Claude-Mem Documentation Folder
|
||||
|
||||
## What This Folder Is
|
||||
|
||||
This `docs/` folder is a **Mintlify documentation site** - the official user-facing documentation for claude-mem. It's a structured documentation platform with a specific file format and organization.
|
||||
|
||||
## File Structure Requirements
|
||||
|
||||
### Mintlify Documentation Files (.mdx)
|
||||
All official documentation files must be:
|
||||
- Written in `.mdx` format (Markdown with JSX support)
|
||||
- Listed in `docs.json` navigation structure
|
||||
- Follow Mintlify's schema and conventions
|
||||
|
||||
The documentation is organized into these sections:
|
||||
- **Get Started**: Introduction, installation, usage guides
|
||||
- **Best Practices**: Context engineering, progressive disclosure
|
||||
- **Configuration & Development**: Settings, dev workflow, troubleshooting
|
||||
- **Architecture**: System design, components, technical details
|
||||
|
||||
### Configuration File
|
||||
`docs.json` defines:
|
||||
- Site metadata (name, description, theme)
|
||||
- Navigation structure
|
||||
- Branding (logos, colors)
|
||||
- Footer links and social media
|
||||
|
||||
## What Does NOT Belong Here
|
||||
|
||||
**Planning documents, design docs, and reference materials should go in `/context/` instead:**
|
||||
|
||||
Files that should be in `/context/` (not `/docs/`):
|
||||
- Planning documents (`*-plan.md`, `*-outline.md`)
|
||||
- Implementation analysis (`*-audit.md`, `*-code-reference.md`)
|
||||
- Error tracking (`typescript-errors.md`)
|
||||
- Design documents not part of official docs
|
||||
- PR review responses
|
||||
- Reference materials (like `agent-sdk-ref.md`)
|
||||
|
||||
**Example**: The deleted `VIEWER.md` was moved because it was implementation documentation, not user-facing docs.
|
||||
|
||||
## Current Files That Should Be Moved
|
||||
|
||||
These `.md` files currently in `docs/` should probably be moved to `context/`:
|
||||
- `typescript-errors.md` - Error tracking
|
||||
- `worker-service-architecture.md` - Implementation details (not user-facing architecture)
|
||||
- `processing-indicator-audit.md` - Implementation audit
|
||||
- `processing-indicator-code-reference.md` - Code reference
|
||||
- `worker-service-rewrite-outline.md` - Planning document
|
||||
- `worker-service-overhead.md` - Analysis document
|
||||
- `CHROMA.md` - Implementation reference (if not user-facing)
|
||||
- `chroma-search-completion-plan.md` - Planning document
|
||||
|
||||
## How to Add Official Documentation
|
||||
|
||||
1. Create a new `.mdx` file in the appropriate subdirectory
|
||||
2. Add the file path to `docs.json` navigation
|
||||
3. Use Mintlify's frontmatter and components
|
||||
4. Follow the existing documentation style
|
||||
|
||||
## Development Workflow
|
||||
|
||||
**For contributors working on claude-mem:**
|
||||
- Read `/CLAUDE.md` in the project root for development instructions
|
||||
- Place planning/design docs in `/context/`
|
||||
- Only add user-facing documentation to `/docs/`
|
||||
- Test documentation locally with Mintlify CLI if available
|
||||
|
||||
## Summary
|
||||
|
||||
**Simple Rule**:
|
||||
- `/docs/` = Official user documentation (Mintlify .mdx files)
|
||||
- `/context/` = Development context, plans, references, internal docs
|
||||
@@ -0,0 +1,104 @@
|
||||
# Claude-Mem Context Documentation
|
||||
|
||||
## What This Folder Is
|
||||
|
||||
This `docs/context/` folder contains **internal documentation** - planning documents, design references, audits, and work-in-progress materials that support development but are NOT user-facing.
|
||||
|
||||
## Folder Structure
|
||||
|
||||
```
|
||||
docs/
|
||||
├── public/ ← User-facing Mintlify docs (DO NOT put internal docs there)
|
||||
│ └── *.mdx - Official documentation
|
||||
└── context/ ← You are here (Internal documentation)
|
||||
├── *.md - Planning docs, audits, references
|
||||
├── *-plan.md - Implementation plans
|
||||
├── *-audit.md - Code audits and reviews
|
||||
├── agent-sdk-*.md - SDK reference materials
|
||||
└── subdirs/ - Organized by topic
|
||||
```
|
||||
|
||||
## What Belongs Here
|
||||
|
||||
**Internal Documentation** (`.md` format):
|
||||
- Planning documents (`*-plan.md`, `*-outline.md`)
|
||||
- Implementation analysis (`*-audit.md`, `*-code-reference.md`)
|
||||
- Error tracking (`typescript-errors.md`)
|
||||
- Design documents not ready for public docs
|
||||
- PR review responses
|
||||
- Reference materials (like `agent-sdk-ref.md`)
|
||||
- Work-in-progress documentation
|
||||
- Technical investigations and postmortems
|
||||
- Architecture analysis documents
|
||||
|
||||
**Examples from this folder:**
|
||||
- `mem-search-technical-architecture.md` - Deep technical reference
|
||||
- `search-architecture-analysis.md` - Implementation analysis
|
||||
- `agent-sdk-ref.md` - SDK reference for developers
|
||||
- `typescript-errors.md` - Error tracking during development
|
||||
- `worker-service-architecture.md` - Internal architecture notes
|
||||
- `processing-indicator-audit.md` - Code audit document
|
||||
|
||||
## What Does NOT Belong Here
|
||||
|
||||
**User-Facing Documentation** goes in `/docs/public/`:
|
||||
- User guides and tutorials
|
||||
- Official architecture documentation
|
||||
- Installation instructions
|
||||
- Configuration guides
|
||||
- Best practices for users
|
||||
- Troubleshooting guides
|
||||
|
||||
**Rule of Thumb:**
|
||||
- If a user would read it → `/docs/public/` (as `.mdx`)
|
||||
- If only developers/contributors need it → `/docs/context/` (as `.md`)
|
||||
|
||||
## File Organization
|
||||
|
||||
### By Type
|
||||
- `*-plan.md` - Implementation plans for features
|
||||
- `*-audit.md` - Code audits and reviews
|
||||
- `*-postmortem.md` - Analysis of issues or incidents
|
||||
- `*-reference.md` - Technical reference materials
|
||||
- `*-analysis.md` - Architecture or design analysis
|
||||
|
||||
### By Topic
|
||||
- Create subdirectories for related documents
|
||||
- Example: `claude-code/` for Claude Code specific docs
|
||||
- Example: `architecture/` for internal architecture notes
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### When to Create Context Docs
|
||||
|
||||
1. **Planning Phase** - Before implementing a feature
|
||||
- Create `feature-name-plan.md`
|
||||
- Outline implementation steps
|
||||
- Document decisions and tradeoffs
|
||||
|
||||
2. **During Development** - Track issues and decisions
|
||||
- Create `feature-name-audit.md` for code reviews
|
||||
- Update `typescript-errors.md` for build issues
|
||||
- Document gotchas in topic-specific files
|
||||
|
||||
3. **After Implementation** - Preserve knowledge
|
||||
- Create `feature-name-postmortem.md` if issues occurred
|
||||
- Update architecture analysis documents
|
||||
- Archive plan docs (don't delete - useful for history)
|
||||
|
||||
### Graduating to Public Docs
|
||||
|
||||
When internal docs are polished enough for users:
|
||||
1. Convert `.md` to `.mdx` format
|
||||
2. Add Mintlify frontmatter
|
||||
3. Move to appropriate `/docs/public/` subdirectory
|
||||
4. Add to `docs.json` navigation
|
||||
5. Keep original in `/docs/context/` for reference
|
||||
|
||||
## Summary
|
||||
|
||||
**Simple Rule**:
|
||||
- `/docs/context/` = Internal docs, plans, references, audits ← YOU ARE HERE
|
||||
- `/docs/public/` = Official user documentation (Mintlify .mdx files)
|
||||
|
||||
**Purpose**: This folder preserves development context, design decisions, and technical knowledge that helps contributors understand WHY things work the way they do, even if users don't need those details.
|
||||
@@ -0,0 +1,164 @@
|
||||
# CWD Context Fix - Technical Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
This fix adds working directory (CWD) context propagation through the entire claude-mem pipeline, enabling the SDK agent to have spatial awareness of which directory/repository it's observing.
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Previously, the SDK agent would:
|
||||
- Search wrong repositories when analyzing file operations
|
||||
- Report "file not found" for files that actually exist
|
||||
- Lack context about which project was being worked on
|
||||
- Generate inaccurate observations due to spatial confusion
|
||||
|
||||
## Solution
|
||||
|
||||
The CWD information now flows through the entire system:
|
||||
|
||||
```
|
||||
Hook Input (cwd) → Worker API (cwd) → SessionManager (cwd) → SDK Agent (tool_cwd)
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
### 1. Hook Layer (`save-hook.ts`)
|
||||
```typescript
|
||||
export interface PostToolUseInput {
|
||||
session_id: string;
|
||||
cwd: string; // ← Captured from Claude Code
|
||||
tool_name: string;
|
||||
tool_input: any;
|
||||
tool_response: any;
|
||||
}
|
||||
```
|
||||
|
||||
The hook extracts `cwd` and includes it in the worker API request:
|
||||
```typescript
|
||||
body: JSON.stringify({
|
||||
tool_name,
|
||||
tool_input,
|
||||
tool_response,
|
||||
prompt_number,
|
||||
cwd: cwd || '' // ← Passed to worker
|
||||
})
|
||||
```
|
||||
|
||||
### 2. Worker Service (`worker-service.ts`)
|
||||
```typescript
|
||||
const { tool_name, tool_input, tool_response, prompt_number, cwd } = req.body;
|
||||
|
||||
this.sessionManager.queueObservation(sessionDbId, {
|
||||
tool_name,
|
||||
tool_input,
|
||||
tool_response,
|
||||
prompt_number,
|
||||
cwd // ← Forwarded to queue
|
||||
});
|
||||
```
|
||||
|
||||
### 3. Session Manager (`SessionManager.ts`)
|
||||
```typescript
|
||||
session.pendingMessages.push({
|
||||
type: 'observation',
|
||||
tool_name: data.tool_name,
|
||||
tool_input: data.tool_input,
|
||||
tool_response: data.tool_response,
|
||||
prompt_number: data.prompt_number,
|
||||
cwd: data.cwd // ← Included in message queue
|
||||
});
|
||||
```
|
||||
|
||||
### 4. SDK Agent (`SDKAgent.ts`)
|
||||
```typescript
|
||||
content: buildObservationPrompt({
|
||||
id: 0,
|
||||
tool_name: message.tool_name!,
|
||||
tool_input: JSON.stringify(message.tool_input),
|
||||
tool_output: JSON.stringify(message.tool_response),
|
||||
created_at_epoch: Date.now(),
|
||||
cwd: message.cwd // ← Passed to prompt builder
|
||||
})
|
||||
```
|
||||
|
||||
### 5. Prompt Generation (`prompts.ts`)
|
||||
```typescript
|
||||
return `<tool_used>
|
||||
<tool_name>${obs.tool_name}</tool_name>
|
||||
<tool_time>${new Date(obs.created_at_epoch).toISOString()}</tool_time>${obs.cwd ? `
|
||||
<tool_cwd>${obs.cwd}</tool_cwd>` : ''} // ← Included in XML
|
||||
<tool_input>${JSON.stringify(toolInput, null, 2)}</tool_input>
|
||||
<tool_output>${JSON.stringify(toolOutput, null, 2)}</tool_output>
|
||||
</tool_used>`;
|
||||
```
|
||||
|
||||
## SDK Agent Prompt Changes
|
||||
|
||||
The init prompt now includes a "SPATIAL AWARENESS" section:
|
||||
|
||||
```
|
||||
SPATIAL AWARENESS: Tool executions include the working directory (tool_cwd) to help you understand:
|
||||
- Which repository/project is being worked on
|
||||
- Where files are located relative to the project root
|
||||
- How to match requested paths to actual execution paths
|
||||
```
|
||||
|
||||
## Example Usage
|
||||
|
||||
When a user executes a read operation in `/home/user/my-project`:
|
||||
|
||||
```xml
|
||||
<tool_used>
|
||||
<tool_name>ReadTool</tool_name>
|
||||
<tool_time>2025-11-10T19:18:03.065Z</tool_time>
|
||||
<tool_cwd>/home/user/my-project</tool_cwd>
|
||||
<tool_input>
|
||||
{
|
||||
"path": "src/index.ts"
|
||||
}
|
||||
</tool_input>
|
||||
<tool_output>
|
||||
{
|
||||
"content": "export default..."
|
||||
}
|
||||
</tool_output>
|
||||
</tool_used>
|
||||
```
|
||||
|
||||
The SDK agent now knows:
|
||||
1. The operation happened in `/home/user/my-project`
|
||||
2. The file `src/index.ts` is relative to that directory
|
||||
3. Which repository context to search when generating observations
|
||||
|
||||
## Testing
|
||||
|
||||
8 comprehensive tests validate the CWD propagation:
|
||||
|
||||
```bash
|
||||
npx tsx --test tests/cwd-propagation.test.ts
|
||||
```
|
||||
|
||||
All tests verify:
|
||||
- Type interfaces include `cwd` fields
|
||||
- Hook extracts and passes `cwd`
|
||||
- Worker accepts and forwards `cwd`
|
||||
- SDK agent includes `cwd` in prompts
|
||||
- End-to-end flow is correct
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Spatial Awareness**: SDK agent knows which directory/repository it's observing
|
||||
2. **Accurate Path Matching**: Can verify if requested paths match executed paths
|
||||
3. **Better Summaries**: Won't search wrong repositories or report false negatives
|
||||
4. **Works with All Models**: Even Haiku benefits from correct context (no need for Opus workaround)
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
- `cwd` is optional in all interfaces (`cwd?: string`)
|
||||
- Missing `cwd` values are handled gracefully (defaults to empty string)
|
||||
- Existing observations without `cwd` continue to work
|
||||
- No database migration required (CWD is transient, not persisted)
|
||||
|
||||
## Related Issues
|
||||
|
||||
Fixes issue #73 (CWD context missing from SDK agent)
|
||||
@@ -0,0 +1,424 @@
|
||||
"""Pydantic models for Claude Code transcript JSON structures.
|
||||
|
||||
Enhanced to leverage official Anthropic types where beneficial.
|
||||
"""
|
||||
|
||||
from typing import Any, List, Union, Optional, Dict, Literal, cast
|
||||
from pydantic import BaseModel
|
||||
|
||||
from anthropic.types import Message as AnthropicMessage
|
||||
from anthropic.types import StopReason
|
||||
from anthropic.types import Usage as AnthropicUsage
|
||||
from anthropic.types.content_block import ContentBlock
|
||||
|
||||
|
||||
class TodoItem(BaseModel):
|
||||
id: str
|
||||
content: str
|
||||
status: Literal["pending", "in_progress", "completed"]
|
||||
priority: Literal["high", "medium", "low"]
|
||||
|
||||
|
||||
class UsageInfo(BaseModel):
|
||||
"""Token usage information that extends Anthropic's Usage type to handle optional fields."""
|
||||
|
||||
input_tokens: Optional[int] = None
|
||||
cache_creation_input_tokens: Optional[int] = None
|
||||
cache_read_input_tokens: Optional[int] = None
|
||||
output_tokens: Optional[int] = None
|
||||
service_tier: Optional[str] = None
|
||||
server_tool_use: Optional[Dict[str, Any]] = None
|
||||
|
||||
def to_anthropic_usage(self) -> Optional[AnthropicUsage]:
|
||||
"""Convert to Anthropic Usage type if both required fields are present."""
|
||||
if self.input_tokens is not None and self.output_tokens is not None:
|
||||
return AnthropicUsage(
|
||||
input_tokens=self.input_tokens,
|
||||
output_tokens=self.output_tokens,
|
||||
cache_creation_input_tokens=self.cache_creation_input_tokens,
|
||||
cache_read_input_tokens=self.cache_read_input_tokens,
|
||||
service_tier=self.service_tier, # type: ignore
|
||||
server_tool_use=self.server_tool_use, # type: ignore
|
||||
)
|
||||
return None
|
||||
|
||||
@classmethod
|
||||
def from_anthropic_usage(cls, usage: AnthropicUsage) -> "UsageInfo":
|
||||
"""Create UsageInfo from Anthropic Usage."""
|
||||
return cls(
|
||||
input_tokens=usage.input_tokens,
|
||||
output_tokens=usage.output_tokens,
|
||||
cache_creation_input_tokens=usage.cache_creation_input_tokens,
|
||||
cache_read_input_tokens=usage.cache_read_input_tokens,
|
||||
service_tier=usage.service_tier,
|
||||
server_tool_use=usage.server_tool_use.model_dump()
|
||||
if usage.server_tool_use
|
||||
else None,
|
||||
)
|
||||
|
||||
|
||||
class TextContent(BaseModel):
|
||||
type: Literal["text"]
|
||||
text: str
|
||||
|
||||
|
||||
class ToolUseContent(BaseModel):
|
||||
type: Literal["tool_use"]
|
||||
id: str
|
||||
name: str
|
||||
input: Dict[str, Any]
|
||||
|
||||
|
||||
class ToolResultContent(BaseModel):
|
||||
type: Literal["tool_result"]
|
||||
tool_use_id: str
|
||||
content: Union[str, List[Dict[str, Any]]]
|
||||
is_error: Optional[bool] = None
|
||||
|
||||
|
||||
class ThinkingContent(BaseModel):
|
||||
type: Literal["thinking"]
|
||||
thinking: str
|
||||
signature: Optional[str] = None
|
||||
|
||||
|
||||
class ImageSource(BaseModel):
|
||||
type: Literal["base64"]
|
||||
media_type: str
|
||||
data: str
|
||||
|
||||
|
||||
class ImageContent(BaseModel):
|
||||
type: Literal["image"]
|
||||
source: ImageSource
|
||||
|
||||
|
||||
# Enhanced ContentItem to include official Anthropic ContentBlock types
|
||||
ContentItem = Union[
|
||||
TextContent,
|
||||
ToolUseContent,
|
||||
ToolResultContent,
|
||||
ThinkingContent,
|
||||
ImageContent,
|
||||
ContentBlock, # Official Anthropic content block types
|
||||
]
|
||||
|
||||
|
||||
class UserMessage(BaseModel):
|
||||
role: Literal["user"]
|
||||
content: Union[str, List[ContentItem]]
|
||||
|
||||
|
||||
class AssistantMessage(BaseModel):
|
||||
"""Assistant message model compatible with Anthropic's Message type."""
|
||||
|
||||
id: str
|
||||
type: Literal["message"]
|
||||
role: Literal["assistant"]
|
||||
model: str
|
||||
content: List[ContentItem]
|
||||
stop_reason: Optional[StopReason] = None
|
||||
stop_sequence: Optional[str] = None
|
||||
usage: Optional[UsageInfo] = None
|
||||
|
||||
@classmethod
|
||||
def from_anthropic_message(
|
||||
cls, anthropic_msg: AnthropicMessage
|
||||
) -> "AssistantMessage":
|
||||
"""Create AssistantMessage from official Anthropic Message."""
|
||||
# Convert Anthropic Message to our format, preserving official types where possible
|
||||
return cls(
|
||||
id=anthropic_msg.id,
|
||||
type=anthropic_msg.type,
|
||||
role=anthropic_msg.role,
|
||||
model=anthropic_msg.model,
|
||||
content=list(
|
||||
anthropic_msg.content
|
||||
), # Convert to list for ContentItem compatibility
|
||||
stop_reason=anthropic_msg.stop_reason,
|
||||
stop_sequence=anthropic_msg.stop_sequence,
|
||||
usage=normalize_usage_info(anthropic_msg.usage),
|
||||
)
|
||||
|
||||
|
||||
class FileInfo(BaseModel):
|
||||
filePath: str
|
||||
content: str
|
||||
numLines: int
|
||||
startLine: int
|
||||
totalLines: int
|
||||
|
||||
|
||||
class FileReadResult(BaseModel):
|
||||
type: Literal["text"]
|
||||
file: FileInfo
|
||||
|
||||
|
||||
class CommandResult(BaseModel):
|
||||
stdout: str
|
||||
stderr: str
|
||||
interrupted: bool
|
||||
isImage: bool
|
||||
|
||||
|
||||
class TodoResult(BaseModel):
|
||||
oldTodos: List[TodoItem]
|
||||
newTodos: List[TodoItem]
|
||||
|
||||
|
||||
class EditResult(BaseModel):
|
||||
oldString: Optional[str] = None
|
||||
newString: Optional[str] = None
|
||||
replaceAll: Optional[bool] = None
|
||||
originalFile: Optional[str] = None
|
||||
structuredPatch: Optional[Any] = None
|
||||
userModified: Optional[bool] = None
|
||||
|
||||
|
||||
ToolUseResult = Union[
|
||||
str,
|
||||
List[TodoItem],
|
||||
FileReadResult,
|
||||
CommandResult,
|
||||
TodoResult,
|
||||
EditResult,
|
||||
List[ContentItem],
|
||||
]
|
||||
|
||||
|
||||
class BaseTranscriptEntry(BaseModel):
|
||||
parentUuid: Optional[str]
|
||||
isSidechain: bool
|
||||
userType: str
|
||||
cwd: str
|
||||
sessionId: str
|
||||
version: str
|
||||
uuid: str
|
||||
timestamp: str
|
||||
isMeta: Optional[bool] = None
|
||||
|
||||
|
||||
class UserTranscriptEntry(BaseTranscriptEntry):
|
||||
type: Literal["user"]
|
||||
message: UserMessage
|
||||
toolUseResult: Optional[ToolUseResult] = None
|
||||
|
||||
|
||||
class AssistantTranscriptEntry(BaseTranscriptEntry):
|
||||
type: Literal["assistant"]
|
||||
message: AssistantMessage
|
||||
requestId: Optional[str] = None
|
||||
|
||||
|
||||
class SummaryTranscriptEntry(BaseModel):
|
||||
type: Literal["summary"]
|
||||
summary: str
|
||||
leafUuid: str
|
||||
cwd: Optional[str] = None
|
||||
|
||||
|
||||
class SystemTranscriptEntry(BaseTranscriptEntry):
|
||||
"""System messages like warnings, notifications, etc."""
|
||||
|
||||
type: Literal["system"]
|
||||
content: str
|
||||
level: Optional[str] = None # e.g., "warning", "info", "error"
|
||||
|
||||
|
||||
class QueueOperationTranscriptEntry(BaseModel):
|
||||
"""Queue operations (enqueue/dequeue) for message queueing tracking.
|
||||
|
||||
These are internal operations that track when messages are queued and dequeued.
|
||||
They are parsed but not rendered, as the content duplicates actual user messages.
|
||||
"""
|
||||
|
||||
type: Literal["queue-operation"]
|
||||
operation: Literal["enqueue", "dequeue"]
|
||||
timestamp: str
|
||||
sessionId: str
|
||||
content: Optional[List[ContentItem]] = None # Only present for enqueue operations
|
||||
|
||||
|
||||
TranscriptEntry = Union[
|
||||
UserTranscriptEntry,
|
||||
AssistantTranscriptEntry,
|
||||
SummaryTranscriptEntry,
|
||||
SystemTranscriptEntry,
|
||||
QueueOperationTranscriptEntry,
|
||||
]
|
||||
|
||||
|
||||
def normalize_usage_info(usage_data: Any) -> Optional[UsageInfo]:
|
||||
"""Normalize usage data to be compatible with both custom and Anthropic formats."""
|
||||
if usage_data is None:
|
||||
return None
|
||||
|
||||
# If it's already a UsageInfo instance, return as-is
|
||||
if isinstance(usage_data, UsageInfo):
|
||||
return usage_data
|
||||
|
||||
# If it's an Anthropic Usage instance, convert using our method
|
||||
if isinstance(usage_data, AnthropicUsage):
|
||||
return UsageInfo.from_anthropic_usage(usage_data)
|
||||
|
||||
# If it has the shape of an Anthropic Usage, try to construct it first
|
||||
if hasattr(usage_data, "input_tokens") and hasattr(usage_data, "output_tokens"):
|
||||
try:
|
||||
# Try to create an Anthropic Usage first
|
||||
anthropic_usage = AnthropicUsage.model_validate(usage_data)
|
||||
return UsageInfo.from_anthropic_usage(anthropic_usage)
|
||||
except Exception:
|
||||
# Fall back to direct conversion
|
||||
return UsageInfo(
|
||||
input_tokens=getattr(usage_data, "input_tokens", None),
|
||||
cache_creation_input_tokens=getattr(
|
||||
usage_data, "cache_creation_input_tokens", None
|
||||
),
|
||||
cache_read_input_tokens=getattr(
|
||||
usage_data, "cache_read_input_tokens", None
|
||||
),
|
||||
output_tokens=getattr(usage_data, "output_tokens", None),
|
||||
service_tier=getattr(usage_data, "service_tier", None),
|
||||
server_tool_use=getattr(usage_data, "server_tool_use", None),
|
||||
)
|
||||
|
||||
# If it's a dict, validate and convert to our format
|
||||
if isinstance(usage_data, dict):
|
||||
return UsageInfo.model_validate(usage_data)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def parse_content_item(item_data: Dict[str, Any]) -> ContentItem:
|
||||
"""Parse a content item using enhanced approach with Anthropic types."""
|
||||
try:
|
||||
content_type = item_data.get("type", "")
|
||||
|
||||
# Try official Anthropic types first for better future compatibility
|
||||
if content_type == "text":
|
||||
try:
|
||||
from anthropic.types.text_block import TextBlock
|
||||
|
||||
return TextBlock.model_validate(item_data)
|
||||
except Exception:
|
||||
return TextContent.model_validate(item_data)
|
||||
elif content_type == "tool_use":
|
||||
try:
|
||||
from anthropic.types.tool_use_block import ToolUseBlock
|
||||
|
||||
return ToolUseBlock.model_validate(item_data)
|
||||
except Exception:
|
||||
return ToolUseContent.model_validate(item_data)
|
||||
elif content_type == "thinking":
|
||||
try:
|
||||
from anthropic.types.thinking_block import ThinkingBlock
|
||||
|
||||
return ThinkingBlock.model_validate(item_data)
|
||||
except Exception:
|
||||
return ThinkingContent.model_validate(item_data)
|
||||
elif content_type == "tool_result":
|
||||
return ToolResultContent.model_validate(item_data)
|
||||
elif content_type == "image":
|
||||
return ImageContent.model_validate(item_data)
|
||||
else:
|
||||
# Fallback to text content for unknown types
|
||||
return TextContent(type="text", text=str(item_data))
|
||||
except Exception:
|
||||
return TextContent(type="text", text=str(item_data))
|
||||
|
||||
|
||||
def parse_message_content(content_data: Any) -> Union[str, List[ContentItem]]:
|
||||
"""Parse message content, handling both string and list formats."""
|
||||
if isinstance(content_data, str):
|
||||
return content_data
|
||||
elif isinstance(content_data, list):
|
||||
content_list = cast(List[Dict[str, Any]], content_data)
|
||||
return [parse_content_item(item) for item in content_list]
|
||||
else:
|
||||
return str(content_data)
|
||||
|
||||
|
||||
def parse_transcript_entry(data: Dict[str, Any]) -> TranscriptEntry:
|
||||
"""
|
||||
Parse a JSON dictionary into the appropriate TranscriptEntry type.
|
||||
|
||||
Enhanced to optionally use official Anthropic types for assistant messages.
|
||||
|
||||
Args:
|
||||
data: Dictionary parsed from JSON
|
||||
|
||||
Returns:
|
||||
The appropriate TranscriptEntry subclass
|
||||
|
||||
Raises:
|
||||
ValueError: If the data doesn't match any known transcript entry type
|
||||
"""
|
||||
entry_type = data.get("type")
|
||||
|
||||
if entry_type == "user":
|
||||
# Parse message content if present
|
||||
data_copy = data.copy()
|
||||
if "message" in data_copy and "content" in data_copy["message"]:
|
||||
data_copy["message"] = data_copy["message"].copy()
|
||||
data_copy["message"]["content"] = parse_message_content(
|
||||
data_copy["message"]["content"]
|
||||
)
|
||||
# Parse toolUseResult if present and it's a list of content items
|
||||
if "toolUseResult" in data_copy and isinstance(
|
||||
data_copy["toolUseResult"], list
|
||||
):
|
||||
# Check if it's a list of content items (MCP tool results)
|
||||
tool_use_result = cast(List[Any], data_copy["toolUseResult"])
|
||||
if (
|
||||
tool_use_result
|
||||
and isinstance(tool_use_result[0], dict)
|
||||
and "type" in tool_use_result[0]
|
||||
):
|
||||
data_copy["toolUseResult"] = [
|
||||
parse_content_item(cast(Dict[str, Any], item))
|
||||
for item in tool_use_result
|
||||
if isinstance(item, dict)
|
||||
]
|
||||
return UserTranscriptEntry.model_validate(data_copy)
|
||||
|
||||
elif entry_type == "assistant":
|
||||
# Enhanced assistant message parsing with optional Anthropic types
|
||||
data_copy = data.copy()
|
||||
|
||||
# Validate compatibility with official Anthropic Message type
|
||||
if "message" in data_copy:
|
||||
try:
|
||||
message_data = data_copy["message"]
|
||||
AnthropicMessage.model_validate(message_data)
|
||||
# Successfully validated - our data is compatible with official Anthropic types
|
||||
except Exception:
|
||||
# Validation failed - continue with standard parsing
|
||||
pass
|
||||
|
||||
# Standard parsing path (works for all cases)
|
||||
if "message" in data_copy and "content" in data_copy["message"]:
|
||||
message_copy = data_copy["message"].copy()
|
||||
message_copy["content"] = parse_message_content(message_copy["content"])
|
||||
|
||||
# Normalize usage data to support both Anthropic and custom formats
|
||||
if "usage" in message_copy:
|
||||
message_copy["usage"] = normalize_usage_info(message_copy["usage"])
|
||||
|
||||
data_copy["message"] = message_copy
|
||||
return AssistantTranscriptEntry.model_validate(data_copy)
|
||||
|
||||
elif entry_type == "summary":
|
||||
return SummaryTranscriptEntry.model_validate(data)
|
||||
|
||||
elif entry_type == "system":
|
||||
return SystemTranscriptEntry.model_validate(data)
|
||||
|
||||
elif entry_type == "queue-operation":
|
||||
# Parse content if present (only in enqueue operations)
|
||||
data_copy = data.copy()
|
||||
if "content" in data_copy and isinstance(data_copy["content"], list):
|
||||
data_copy["content"] = parse_message_content(data_copy["content"])
|
||||
return QueueOperationTranscriptEntry.model_validate(data_copy)
|
||||
|
||||
else:
|
||||
raise ValueError(f"Unknown transcript entry type: {entry_type}")
|
||||
@@ -0,0 +1,616 @@
|
||||
# JIT Context Filtering: Post-Mortem
|
||||
|
||||
**Date:** November 9, 2025
|
||||
**Duration:** 3.5 hours (7:45 PM - 11:11 PM)
|
||||
**Branches:** `feature/jit-context`, `failed/jit-context`
|
||||
**Status:** Failed, reverted to main
|
||||
**Commits:**
|
||||
- `3ac0790` - feat: Implement JIT context hook for user prompt submission
|
||||
- `adf7bf4` - Refactor JIT context handling in SDKAgent and WorkerService
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Attempted to implement JIT (Just-In-Time) context filtering—a feature that would dynamically generate relevant context timelines on every user prompt, potentially replacing the static session-start context entirely. After multiple architectural iterations spanning 3.5 hours and adding ~2,850 lines of code, the implementation was abandoned and reverted. The revert was not due to lack of vision (the feature aligns with long-term architectural goals), but due to implementation complexity and the need for a simpler initial approach. Significant architectural knowledge was gained about hook limitations, worker patterns, and proper separation of concerns.
|
||||
|
||||
## What We Tried to Build
|
||||
|
||||
### Goal
|
||||
When a user submits a prompt, dynamically generate a relevant context timeline instead of the static session-start context. Use the fast search infrastructure (SQLite FTS5 + ChromaDB) to fetch precisely relevant context on-demand.
|
||||
|
||||
### The Vision
|
||||
**Current approach:** SessionStart hook loads 50 recent observations blindly, displays them all.
|
||||
|
||||
**Proposed approach:** UserPromptSubmit hook analyzes the prompt, queries the timeline search API, and loads only the relevant context window dynamically.
|
||||
|
||||
**Why this makes sense:**
|
||||
- We already have fast search: SQLite FTS5 + Chroma semantic search
|
||||
- Dynamic context timeline search is implemented and tested
|
||||
- Search results come back in <200ms
|
||||
- Could **replace** session-start context entirely with smarter, prompt-specific context
|
||||
|
||||
### User Experience
|
||||
```
|
||||
User types: "How did we fix the authentication bug?"
|
||||
|
||||
Behind the scenes:
|
||||
1. Analyze prompt: "authentication bug fix"
|
||||
2. Query timeline search for relevant period
|
||||
3. Load 5-10 observations from that specific timeline
|
||||
4. Inject as context
|
||||
5. Claude answers with precisely relevant historical context
|
||||
|
||||
vs. Current:
|
||||
Load 50 most recent observations regardless of relevance
|
||||
```
|
||||
|
||||
### Why Checkbox Settings Became Less Important
|
||||
Originally asked for checkboxes to customize session-start context display. But if JIT context could replace session-start context with intelligent, prompt-specific timelines, the display customization became a non-issue.
|
||||
|
||||
## Architectural Attempts
|
||||
|
||||
### Attempt 1: Hook-Based Filtering (7:45 PM - 9:30 PM)
|
||||
|
||||
**Approach:** Call Agent SDK `query()` directly in `new-hook.ts` during UserPromptSubmit event.
|
||||
|
||||
**Implementation:**
|
||||
- Created `jit-context-hook.ts` (~432 lines)
|
||||
- Added `generateJitContext()` function in hook
|
||||
- Called SDK `query()` with observation list and user prompt
|
||||
- Expected hook to block for ~1-2s while Haiku filters
|
||||
|
||||
**Failure:**
|
||||
```
|
||||
Error: Claude Code executable not found at
|
||||
/Users/alexnewman/.claude/plugins/marketplaces/thedotmack/plugin/scripts/cli.js
|
||||
```
|
||||
|
||||
**Root Cause:** Hooks run in sandboxed environment without access to `claudePath` (path to Claude Code executable). The Agent SDK requires this path, which is only available in the worker service.
|
||||
|
||||
**Architectural Violation:** This broke the established pattern where hooks handle orchestration and workers handle AI processing. The `save-hook` sets the precedent: hooks capture data, send to worker, worker runs SDK queries asynchronously.
|
||||
|
||||
### Attempt 2: Worker-Based with Simple Queries (9:30 PM - 10:30 PM)
|
||||
|
||||
**Approach:** Move JIT filtering to worker service, keep it simple with per-request SDK queries.
|
||||
|
||||
**Implementation:**
|
||||
- Documented architecture fix plan in `docs/jit-context-architecture-fix.md`
|
||||
- Moved `generateJitContext()` to worker (considered creating `src/services/worker/JitContext.ts`)
|
||||
- Modified `/sessions/:id/init` endpoint to accept `jitEnabled` flag
|
||||
- Worker would run one-shot SDK query per prompt
|
||||
|
||||
**Architecture:**
|
||||
```
|
||||
UserPromptSubmit → new-hook → POST /sessions/:id/init { jitEnabled: true }
|
||||
↓
|
||||
Worker spawns Claude Haiku
|
||||
↓
|
||||
Filters 50 obs → 3-5 IDs
|
||||
↓
|
||||
Returns { context: [...] }
|
||||
↓
|
||||
Hook injects context → Claude
|
||||
```
|
||||
|
||||
**Issues Identified:**
|
||||
- Each filter request spawns a new Claude subprocess (~200-500ms overhead)
|
||||
- Observation list re-sent on every prompt (~5-10KB per request)
|
||||
- No token caching between requests
|
||||
- Performance worse than just loading all observations directly
|
||||
|
||||
**Decision:** Pivoted to persistent sessions to solve performance issues.
|
||||
|
||||
### Attempt 3: Persistent JIT Sessions (10:30 PM - 11:11 PM)
|
||||
|
||||
**Approach:** Create a long-lived Agent SDK session that persists throughout user session, similar to main memory session pattern.
|
||||
|
||||
**Implementation (291 new lines in SDKAgent.ts):**
|
||||
|
||||
1. **Session Lifecycle:**
|
||||
- Added `jitSessionId`, `jitAbortController`, `jitGeneratorPromise` to `ActiveSession` interface
|
||||
- `startJitSession()`: Creates persistent SDK session at session init
|
||||
- `cleanupJitSession()`: Terminates JIT session at session end
|
||||
|
||||
2. **Request Queue Architecture:**
|
||||
- `jitFilterQueues` Map: Per-session request queues
|
||||
- `JITFilterRequest` interface: `{ userPrompt, resolve, reject }`
|
||||
- EventEmitter coordination: Wake generator when new requests arrive
|
||||
|
||||
3. **Message Generator Pattern:**
|
||||
- `createJitMessageGenerator()`: Async generator that yields filter requests
|
||||
- Initial prompt: Load 50 observations, wait for "READY" response
|
||||
- Loop: Wait for EventEmitter signal → yield user prompt → parse response → resolve promise
|
||||
- Pattern: Persistent session stays alive between requests
|
||||
|
||||
4. **Filter Query Flow:**
|
||||
```typescript
|
||||
runFilterQuery(sessionDbId, userPrompt) {
|
||||
// Queue request
|
||||
queue.requests.push({ userPrompt, resolve, reject });
|
||||
queue.emitter.emit('request');
|
||||
|
||||
// Wait for response (30s timeout)
|
||||
return Promise.race([
|
||||
new Promise((resolve, reject) => { /* queued */ }),
|
||||
timeout(30000)
|
||||
]);
|
||||
}
|
||||
```
|
||||
|
||||
5. **Response Processing:**
|
||||
- `processJitFilterResponse()`: Accumulate streaming text
|
||||
- Parse IDs: "1,5,23,41" or "NONE"
|
||||
- Resolve queued promise with ID array
|
||||
|
||||
**Added Files:**
|
||||
- `src/services/worker/SDKAgent.ts`: +291 lines
|
||||
- `src/services/worker-types.ts`: +3 fields (jit state tracking)
|
||||
- `src/services/worker/SessionManager.ts`: +26 lines (JIT cleanup)
|
||||
- `src/services/worker-service.ts`: +102 lines (JIT initialization)
|
||||
- `src/shared/settings.ts`: +65 lines (JIT config)
|
||||
- `src/hooks/jit-context-hook.ts`: +208 lines (orchestration)
|
||||
- `docs/jit-context-architecture-fix.md`: +265 lines
|
||||
- `context/session-pattern-parity.md`: +298 lines
|
||||
|
||||
**Total Changes:** 18 files, +2,852 lines, -133 lines
|
||||
|
||||
**Final Status at Revert:** Implementation was complete and likely functional, but...
|
||||
|
||||
## Why It Failed
|
||||
|
||||
### 1. Architectural Complexity Explosion
|
||||
|
||||
**Problem:** The persistent session pattern added enormous complexity for marginal benefit.
|
||||
|
||||
**Evidence:**
|
||||
- Parallel session management: Regular + JIT sessions running concurrently
|
||||
- Complex coordination: EventEmitter + promise queues + generator pattern
|
||||
- Lifecycle coupling: Session init, request handling, cleanup all intertwined
|
||||
- State explosion: 3 new fields per session (`jitSessionId`, `jitAbortController`, `jitGeneratorPromise`)
|
||||
|
||||
**Code Smell:** When the "optimization" requires 300 lines of coordination code, it's probably not an optimization.
|
||||
|
||||
### 2. Premature Optimization
|
||||
|
||||
**YAGNI Violation:** Built elaborate token caching and persistent session architecture before proving the feature provided value.
|
||||
|
||||
**Reality Check:**
|
||||
- **Current approach:** Load 50 observations = ~25KB context, works fine
|
||||
- **JIT overhead:** Haiku query = 1-2s latency + coordination complexity
|
||||
- **User benefit:** Unclear—users haven't complained about context relevance
|
||||
- **Token savings:** Marginal—Claude caches long contexts efficiently anyway
|
||||
|
||||
**Quote from CLAUDE.md:**
|
||||
> "Write the dumb, obvious thing first. Add complexity only when you actually hit the problem."
|
||||
|
||||
We didn't hit a problem. We invented one.
|
||||
|
||||
### 3. Implementation Complexity, Not Vision
|
||||
|
||||
**The Vision is Sound:**
|
||||
- Dynamic context is better than static context
|
||||
- Timeline search API exists and is fast
|
||||
- Infrastructure (SQLite + Chroma) can support this
|
||||
- Replacing session-start context with prompt-specific context makes sense
|
||||
|
||||
**The Problem:**
|
||||
We jumped to the complex persistent-session approach without trying the simple per-request approach first.
|
||||
|
||||
**What We Should Have Done:**
|
||||
```typescript
|
||||
// Simple version (not tried):
|
||||
app.post('/sessions/:id/init', async (req, res) => {
|
||||
const { userPrompt } = req.body;
|
||||
|
||||
// Query timeline search API (already exists, fast)
|
||||
const timeline = await timelineSearch(project, userPrompt, depth=10);
|
||||
|
||||
// Return observations
|
||||
return res.json({ context: timeline });
|
||||
});
|
||||
```
|
||||
|
||||
**This would have:**
|
||||
- Validated the feature's value quickly
|
||||
- Used existing infrastructure
|
||||
- Avoided all the persistence complexity
|
||||
- Taken 30 minutes instead of 3.5 hours
|
||||
|
||||
### 4. Pattern Divergence
|
||||
|
||||
**Inconsistency:** JIT sessions work fundamentally differently from memory sessions.
|
||||
|
||||
**Memory Session Pattern:**
|
||||
```typescript
|
||||
// One-shot: Init → Process observations → Complete
|
||||
startSession() → yield prompts → parse responses → complete
|
||||
```
|
||||
|
||||
**JIT Session Pattern:**
|
||||
```typescript
|
||||
// Persistent: Init → Wait indefinitely → Process on-demand → Complete
|
||||
startJitSession() → yield initial load → LOOP:
|
||||
- Wait for EventEmitter signal
|
||||
- Yield filter request
|
||||
- Parse response
|
||||
- Resolve promise
|
||||
- GOTO LOOP
|
||||
```
|
||||
|
||||
**Maintenance Burden:** Two completely different session patterns means:
|
||||
- Doubled testing complexity
|
||||
- Increased cognitive load for contributors
|
||||
- Higher risk of subtle bugs in lifecycle management
|
||||
|
||||
**Session Pattern Parity Document:** The 298-line `session-pattern-parity.md` was created to document the differences—a sign that maybe they shouldn't be different.
|
||||
|
||||
### 5. Blocking I/O in Critical Path
|
||||
|
||||
**Performance Impact:** Every user prompt now blocks for 1-2s waiting for Haiku filtering.
|
||||
|
||||
**Current Flow:**
|
||||
```
|
||||
User types prompt → 10ms → Claude responds
|
||||
```
|
||||
|
||||
**JIT Flow:**
|
||||
```
|
||||
User types prompt → 10ms init → 1-2s Haiku filter → Claude responds
|
||||
```
|
||||
|
||||
**User Experience:** We added 1-2 seconds of latency to every interaction for questionable benefit.
|
||||
|
||||
**Alternative:** If context filtering is valuable, do it asynchronously and apply to next prompt.
|
||||
|
||||
### 6. Missing the Forest for the Trees
|
||||
|
||||
**Real Issue:** We focused on technical implementation without asking strategic questions:
|
||||
|
||||
- **Is context relevance actually a problem?** No evidence.
|
||||
- **Do users want this?** No feedback requested.
|
||||
- **Is 50 observations too many?** Not proven.
|
||||
- **Does filtering improve responses?** Not tested.
|
||||
|
||||
**Anti-Pattern:** Solution in search of a problem.
|
||||
|
||||
## What We Should Have Done
|
||||
|
||||
### Option 1: Don't Build It
|
||||
|
||||
**Justification:** No validated user need. Current system works fine.
|
||||
|
||||
**Next Step:** Wait for user feedback indicating context relevance is an issue.
|
||||
|
||||
### Option 2: Simple MVP
|
||||
|
||||
If we really wanted to explore this:
|
||||
|
||||
1. **Week 1:** Add basic filtering in worker with one-shot queries
|
||||
- Accept slight performance hit (~500ms overhead)
|
||||
- Measure filter accuracy and user impact
|
||||
- Gather feedback
|
||||
|
||||
2. **Week 2:** If proven valuable, optimize
|
||||
- Add token caching only if needed
|
||||
- Consider persistent sessions only if performance is bottleneck
|
||||
|
||||
3. **Week 3:** If still valuable, scale
|
||||
- Polish error handling
|
||||
- Add configuration options
|
||||
- Document patterns
|
||||
|
||||
**Philosophy:** Incremental validation, not big-bang architecture.
|
||||
|
||||
### Option 3: Different Approach Entirely
|
||||
|
||||
**Alternative:** Pre-computed relevance scores
|
||||
|
||||
Instead of on-demand filtering:
|
||||
- Score observations at creation time (save-hook)
|
||||
- Store relevance embeddings in Chroma
|
||||
- At session start, query Chroma with user's first prompt
|
||||
- Load top 10-20 most relevant observations
|
||||
- No runtime latency, better accuracy, simpler architecture
|
||||
|
||||
**Benefit:** Leverages existing Chroma infrastructure, avoids runtime overhead.
|
||||
|
||||
## Technical Lessons Learned
|
||||
|
||||
### 1. EventEmitter Coordination Anti-Pattern
|
||||
|
||||
**Code:**
|
||||
```typescript
|
||||
queue.emitter.on('request', () => {
|
||||
// Wake up generator to process request
|
||||
});
|
||||
```
|
||||
|
||||
**Issue:** Complex async coordination using event-driven wakeup signals is hard to reason about.
|
||||
|
||||
**Better:** Use async queues or channels (e.g., `async-queue` package) that handle coordination internally.
|
||||
|
||||
### 2. Generator Pattern Complexity
|
||||
|
||||
**Pattern:**
|
||||
```typescript
|
||||
async *createJitMessageGenerator() {
|
||||
yield initialPrompt;
|
||||
while (!aborted) {
|
||||
await waitForEvent(); // Blocks here
|
||||
yield nextRequest;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Tradeoff:** Generators are great for iteration, but terrible for event-driven request/response patterns.
|
||||
|
||||
**Better:** Use explicit session object with `sendMessage()/waitForResponse()` methods.
|
||||
|
||||
### 3. Dual Session Management
|
||||
|
||||
**Complexity:** Managing two concurrent SDK sessions per user session is inherently complex.
|
||||
|
||||
**Alternatives Considered:**
|
||||
- Single session handling both observations and filtering (rejected: tight coupling)
|
||||
- Separate service for filtering (rejected: too much infrastructure)
|
||||
- Pre-computed filtering (not considered: should have been)
|
||||
|
||||
**Lesson:** When parallel state management feels hard, question whether you need parallel state.
|
||||
|
||||
### 4. Promise Queue Pattern
|
||||
|
||||
**Implementation:**
|
||||
```typescript
|
||||
interface QueuedRequest {
|
||||
resolve: (result: T) => void;
|
||||
reject: (error: Error) => void;
|
||||
}
|
||||
queue.push({ resolve, reject });
|
||||
// Later...
|
||||
queue[0].resolve(result);
|
||||
```
|
||||
|
||||
**Good:** Clean async API for callers
|
||||
**Bad:** Easy to leak promises if error handling isn't perfect
|
||||
**Improvement:** Use libraries like `p-queue` that handle edge cases
|
||||
|
||||
## Process Lessons Learned
|
||||
|
||||
### 1. No Incremental Validation
|
||||
|
||||
**Mistake:** Went from "idea" to "complete architecture" without validation points.
|
||||
|
||||
**Better Process:**
|
||||
1. Write one-pager explaining user value
|
||||
2. Build simplest possible version (2 hours max)
|
||||
3. Test with real usage
|
||||
4. Measure impact
|
||||
5. Decide: kill, iterate, or scale
|
||||
|
||||
**Checkpoint Questions:**
|
||||
- After 1 hour: "Does this solve a real problem?"
|
||||
- After 2 hours: "Is this getting too complex?"
|
||||
- After 3 hours: "Should I just ship the simple version?"
|
||||
|
||||
### 2. Architecture Astronomy
|
||||
|
||||
**Definition:** Designing elaborate systems without building/testing them.
|
||||
|
||||
**Evidence:**
|
||||
- 265-line architecture doc written before any code
|
||||
- 298-line session pattern parity analysis
|
||||
- Multiple complete rewrites of the same feature
|
||||
|
||||
**Better:** Code first, document later. Spike solutions, learn from implementation.
|
||||
|
||||
### 3. Sunk Cost Fallacy
|
||||
|
||||
**Timeline:**
|
||||
- **Hour 1:** "This seems complex but achievable"
|
||||
- **Hour 2:** "We're halfway done, can't stop now"
|
||||
- **Hour 3:** "Just need to fix this one coordination issue"
|
||||
- **Hour 4:** "It's working, but... this feels wrong"
|
||||
|
||||
**Correct Decision:** Revert. Took courage to throw away 4 hours of work.
|
||||
|
||||
**Learning:** Time invested is not a reason to continue. Quality of outcome matters more.
|
||||
|
||||
### 4. Missing User Feedback Loop
|
||||
|
||||
**No User Input:**
|
||||
- Didn't ask: "Is context relevance a problem for you?"
|
||||
- Didn't test: "Does filtered context improve your responses?"
|
||||
- Didn't measure: "Are you hitting context limits?"
|
||||
|
||||
**Engineering Theater:** Building impressive-sounding features without user validation.
|
||||
|
||||
## What We Actually Learned (The Real Value)
|
||||
|
||||
Despite reverting, this was productive R&D:
|
||||
|
||||
### 1. Deep Understanding of Hook Architecture
|
||||
|
||||
**Critical Discovery:** Hooks run in sandboxed environment without `claudePath`.
|
||||
- Hooks cannot call Agent SDK `query()` directly
|
||||
- All AI processing must happen in worker service
|
||||
- This architectural constraint is now documented
|
||||
|
||||
**Learned Pattern:**
|
||||
```
|
||||
Hook (orchestration) → Worker (AI processing)
|
||||
✓ save-hook: Captures data → Worker processes with SDK
|
||||
✓ new-hook: Creates session → Worker returns confirmation
|
||||
✗ jit-hook: Tried SDK in hook → Failed, no claudePath
|
||||
```
|
||||
|
||||
**Value:** Future features will avoid this mistake. We now know the boundary.
|
||||
|
||||
### 2. Worker Architecture Patterns
|
||||
|
||||
**Blocking vs. Non-Blocking:**
|
||||
- SessionStart: Can be non-blocking (context loads async)
|
||||
- UserPromptSubmit: Must be blocking (session must exist before processing)
|
||||
- JIT Context: Must be blocking (context needed before prompt processed)
|
||||
|
||||
**Established Pattern:**
|
||||
```typescript
|
||||
// Worker endpoint for features requiring AI
|
||||
app.post('/sessions/:id/operation', async (req, res) => {
|
||||
const { operationData } = req.body;
|
||||
const result = await sdkAgent.performOperation(operationData);
|
||||
return res.json({ result });
|
||||
});
|
||||
```
|
||||
|
||||
### 3. Persistent Session Management
|
||||
|
||||
**Architecture Knowledge Gained:**
|
||||
- How to maintain long-lived SDK sessions
|
||||
- EventEmitter coordination patterns for request/response
|
||||
- Promise queue management for async operations
|
||||
- Proper cleanup with AbortControllers
|
||||
|
||||
**Pattern Documented:**
|
||||
- Dual session management (regular + JIT)
|
||||
- Generator-based message loops
|
||||
- Request queuing with timeouts
|
||||
|
||||
**Value:** When we build the simpler version, we'll know these patterns.
|
||||
|
||||
### 4. Configuration Infrastructure
|
||||
|
||||
`src/shared/settings.ts` (65 lines) provides reusable configuration patterns:
|
||||
```typescript
|
||||
export function getConfigValue(key: string, defaultValue: string): string {
|
||||
// Priority: settings.json → env var → default
|
||||
}
|
||||
```
|
||||
|
||||
**Kept After Revert:** This module is useful for other features.
|
||||
|
||||
### 5. Key Architectural Decisions Made
|
||||
|
||||
**Decisions that will guide future implementation:**
|
||||
1. JIT context filtering must happen in worker (proven via failed hook attempt)
|
||||
2. Context must be blocking on UserPromptSubmit (session needs context before processing)
|
||||
3. Dynamic timeline search is the right approach (fast, precise, leverages existing infrastructure)
|
||||
4. Simple per-request queries should be tried before persistent sessions
|
||||
|
||||
### 6. Documentation Quality
|
||||
|
||||
- `jit-context-architecture-fix.md`: Documents why hooks can't run SDK queries
|
||||
- `session-pattern-parity.md`: Reference for implementing dual sessions
|
||||
- Hooks reference: Comprehensive hook documentation added
|
||||
|
||||
**Value:** These docs help future contributors understand the system constraints.
|
||||
|
||||
### 7. Infrastructure Validation
|
||||
|
||||
**Confirmed that our search stack is ready:**
|
||||
- SQLite FTS5: Fast full-text search (<50ms)
|
||||
- ChromaDB: Semantic search (<200ms with 8,000+ vectors)
|
||||
- Timeline search API: Already implemented and tested
|
||||
- Worker service: Can handle synchronous AI operations
|
||||
|
||||
**The infrastructure exists. We just need a simpler integration.**
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Immediate Actions
|
||||
|
||||
1. **Archive the work:**
|
||||
- Keep `failed/jit-context` branch for reference
|
||||
- Extract reusable components (settings.ts)
|
||||
- Save architecture docs for future features
|
||||
|
||||
2. **Document the anti-patterns:**
|
||||
- Add this post-mortem to CLAUDE.md references
|
||||
- Update coding standards with lessons learned
|
||||
|
||||
3. **Reset focus:**
|
||||
- Return to validated user needs
|
||||
- Prioritize features with clear value propositions
|
||||
|
||||
### Future Feature Development
|
||||
|
||||
**Gating Questions (Answer before coding):**
|
||||
|
||||
1. **User Value:** What specific user problem does this solve?
|
||||
2. **Evidence:** Have users requested this or reported the underlying issue?
|
||||
3. **Measurement:** How will we know if it's successful?
|
||||
4. **Simplicity:** What's the dumbest version that could work?
|
||||
5. **Time Limit:** If we can't prove value in 2 hours, should we build it?
|
||||
|
||||
**Process:**
|
||||
|
||||
```
|
||||
VALIDATE → BUILD SIMPLE → TEST → MEASURE → DECIDE
|
||||
↑ ↓
|
||||
└──────────── ITERATE OR KILL ────────────┘
|
||||
```
|
||||
|
||||
### If Context Filtering Returns
|
||||
|
||||
Should we revisit this idea in the future:
|
||||
|
||||
**Prerequisites:**
|
||||
- User feedback requesting better context relevance
|
||||
- Metrics showing current context is too broad
|
||||
- Evidence that filtering improves response quality
|
||||
|
||||
**Simple Approach:**
|
||||
```typescript
|
||||
// In worker-service.ts /sessions/:id/init
|
||||
if (jitEnabled) {
|
||||
const observations = await db.getRecentObservations(project, 50);
|
||||
const filtered = await simpleFilter(observations, userPrompt); // One-shot query
|
||||
return { context: filtered };
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- <100 lines of code
|
||||
- <500ms latency impact
|
||||
- No new session types
|
||||
- Degrades gracefully on errors
|
||||
|
||||
**If that works:** Then consider optimization.
|
||||
|
||||
## Conclusion
|
||||
|
||||
JIT context filtering failed not because the vision was wrong, but because we jumped to the complex implementation without validating the simple one first. The feature aligns with long-term goals (dynamic, prompt-specific context using our fast search infrastructure), but the persistent-session architecture was premature optimization.
|
||||
|
||||
**The right call:** Revert the complex implementation. Build the simple version when ready.
|
||||
|
||||
**Key Takeaway:** The vision is sound. The execution was overcomplicated. We now have:
|
||||
- Deep knowledge of hook/worker architecture constraints
|
||||
- Documented patterns for persistent SDK sessions
|
||||
- Validated fast search infrastructure
|
||||
- Clear understanding of what to build next time (simple timeline search API integration)
|
||||
|
||||
**This was R&D, not failure.** We learned what doesn't work (SDK in hooks), what does work (worker-based AI processing), and how to approach it next time (simple API calls before persistent sessions).
|
||||
|
||||
**Next Implementation:**
|
||||
When we revisit this (and we should), start with:
|
||||
1. Worker endpoint that accepts prompt
|
||||
2. Queries existing timeline search API
|
||||
3. Returns context
|
||||
4. Hook injects context
|
||||
5. Validate it improves responses
|
||||
6. Then optimize if needed
|
||||
|
||||
**Final Thought:** Sometimes you have to build the wrong thing to understand the right thing. That's R&D.
|
||||
|
||||
---
|
||||
|
||||
**Branch Status:**
|
||||
- `feature/jit-context`: Abandoned
|
||||
- `failed/jit-context`: Archived for reference
|
||||
- `main`: Stable at v5.4.0
|
||||
|
||||
**Files to Keep:**
|
||||
- `src/shared/settings.ts`: Reusable config utilities
|
||||
|
||||
**Files Discarded:**
|
||||
- Everything else (+2,850 lines)
|
||||
|
||||
**Emotional State:** Relieved. Dodged a maintenance nightmare.
|
||||
@@ -0,0 +1,156 @@
|
||||
# Rich Context Examples
|
||||
|
||||
This document shows what contextual data is available in transcripts
|
||||
that could improve observation generation quality.
|
||||
|
||||
## Statistics
|
||||
|
||||
- Total entries: 369
|
||||
- User messages: 74
|
||||
- Assistant messages: 133
|
||||
- Token usage: 67,465 total
|
||||
- Cache efficiency: 6,979,410 tokens read from cache
|
||||
|
||||
## Conversation Flow
|
||||
|
||||
This shows how user requests, assistant reasoning, and tool executions flow together.
|
||||
This is the rich context currently missing from individual tool observations.
|
||||
|
||||
---
|
||||
|
||||
### Example 1
|
||||
|
||||
#### 👤 User Request
|
||||
```
|
||||
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
|
||||
```
|
||||
|
||||
#### 🔧 Tools Executed (1)
|
||||
|
||||
**TodoWrite**
|
||||
```json
|
||||
{
|
||||
"todos": [
|
||||
{
|
||||
"content": "Read the ROI implementation plan to understand full scope",
|
||||
"status": "in_progress",
|
||||
"activeForm": "Reading ROI implementation plan"
|
||||
},
|
||||
{
|
||||
|
||||
```
|
||||
|
||||
**📊 Data Available for This Exchange:**
|
||||
- User intent: ✅ (218 chars)
|
||||
- Assistant reasoning: ✅ (0 chars)
|
||||
- Thinking process: ❌
|
||||
- Tool executions: ✅ (1 tools)
|
||||
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
|
||||
|
||||
---
|
||||
|
||||
### Example 2
|
||||
|
||||
#### 👤 User Request
|
||||
```
|
||||
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
|
||||
```
|
||||
|
||||
#### 🔧 Tools Executed (1)
|
||||
|
||||
**Glob**
|
||||
- Pattern: `**/*roi*`
|
||||
|
||||
**📊 Data Available for This Exchange:**
|
||||
- User intent: ✅ (218 chars)
|
||||
- Assistant reasoning: ✅ (0 chars)
|
||||
- Thinking process: ❌
|
||||
- Tool executions: ✅ (1 tools)
|
||||
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
|
||||
|
||||
---
|
||||
|
||||
### Example 3
|
||||
|
||||
#### 👤 User Request
|
||||
```
|
||||
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
|
||||
```
|
||||
|
||||
#### 🔧 Tools Executed (1)
|
||||
|
||||
**Glob**
|
||||
- Pattern: `**/*implementation*plan*`
|
||||
|
||||
**📊 Data Available for This Exchange:**
|
||||
- User intent: ✅ (218 chars)
|
||||
- Assistant reasoning: ✅ (0 chars)
|
||||
- Thinking process: ❌
|
||||
- Tool executions: ✅ (1 tools)
|
||||
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
|
||||
|
||||
---
|
||||
|
||||
### Example 4
|
||||
|
||||
#### 👤 User Request
|
||||
```
|
||||
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
|
||||
```
|
||||
|
||||
#### 🔧 Tools Executed (1)
|
||||
|
||||
**Read**
|
||||
- Reading: `/Users/alexnewman/Scripts/claude-mem/docs/context/transcript-data-discovery.md`
|
||||
|
||||
**📊 Data Available for This Exchange:**
|
||||
- User intent: ✅ (218 chars)
|
||||
- Assistant reasoning: ✅ (0 chars)
|
||||
- Thinking process: ❌
|
||||
- Tool executions: ✅ (1 tools)
|
||||
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
|
||||
|
||||
---
|
||||
|
||||
### Example 5
|
||||
|
||||
#### 👤 User Request
|
||||
```
|
||||
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
|
||||
```
|
||||
|
||||
#### 🔧 Tools Executed (1)
|
||||
|
||||
**Read**
|
||||
- Reading: `/Users/alexnewman/Scripts/claude-mem/IMPLEMENTATION_PLAN_ROI_METRICS.md`
|
||||
|
||||
**📊 Data Available for This Exchange:**
|
||||
- User intent: ✅ (218 chars)
|
||||
- Assistant reasoning: ✅ (0 chars)
|
||||
- Thinking process: ❌
|
||||
- Tool executions: ✅ (1 tools)
|
||||
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Key Insight
|
||||
|
||||
Currently, the memory worker receives **isolated tool executions** via save-hook:
|
||||
- tool_name: "Read"
|
||||
- tool_input: {"file_path": "src/foo.ts"}
|
||||
- tool_output: {file contents}
|
||||
|
||||
But the transcript contains **rich contextual data**:
|
||||
- WHY the tool was used (user's request)
|
||||
- WHAT the assistant planned to accomplish
|
||||
- HOW it fits into the broader task
|
||||
- The assistant's reasoning/thinking
|
||||
- Multiple related tools used together
|
||||
|
||||
This context would help the memory worker:
|
||||
1. Understand if a tool use is meaningful or routine
|
||||
2. Generate observations that capture WHY, not just WHAT
|
||||
3. Group related tools into coherent actions
|
||||
4. Avoid "investigating" - the context is already present
|
||||
|
||||
@@ -0,0 +1,857 @@
|
||||
# Search Architecture Analysis
|
||||
|
||||
**Date:** 2025-11-11 **Scope:** HTTP API endpoints, MCP search server, DRY violations, architectural recommendations
|
||||
|
||||
---
|
||||
|
||||
## Current State: Dual Search Architectures
|
||||
|
||||
### Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Claude Code Session │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────┐ │
|
||||
│ │ mem-search Skill (ACTIVE) │ │
|
||||
│ │ - Uses HTTP API via curl commands │ │
|
||||
│ │ - 10 search operations │ │
|
||||
│ │ - Progressive disclosure workflow │ │
|
||||
│ └────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ │ HTTP GET │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────┐ │
|
||||
│ │ MCP Search Server (DEPRECATED but BUILT) │ │
|
||||
│ │ - .mcp.json configured │ │
|
||||
│ │ - search-server.mjs exists (74KB) │ │
|
||||
│ │ - 9 MCP tools defined │ │
|
||||
│ │ - Not used by skill │ │
|
||||
│ └────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
┌───────────┴───────────┐
|
||||
▼ ▼
|
||||
┌──────────────────────────┐ ┌──────────────────────────┐
|
||||
│ Worker Service │ │ MCP Server │
|
||||
│ (worker-service.ts) │ │ (search-server.ts) │
|
||||
│ │ │ │
|
||||
│ 10 HTTP Endpoints: │ │ 9 MCP Tools: │
|
||||
│ ├─ /api/search/ │ │ ├─ search_observations │
|
||||
│ │ observations │ │ ├─ search_sessions │
|
||||
│ ├─ /api/search/ │ │ ├─ search_user_prompts │
|
||||
│ │ sessions │ │ ├─ find_by_concept │
|
||||
│ ├─ /api/search/ │ │ ├─ find_by_file │
|
||||
│ │ prompts │ │ ├─ find_by_type │
|
||||
│ ├─ /api/search/ │ │ ├─ get_recent_context │
|
||||
│ │ by-concept │ │ ├─ get_context_timeline │
|
||||
│ ├─ /api/search/ │ │ └─ get_timeline_by_query│
|
||||
│ │ by-file │ │ │
|
||||
│ ├─ /api/search/ │ │ Built: ✅ │
|
||||
│ │ by-type │ │ Used: ❌ │
|
||||
│ ├─ /api/context/recent │ │ Configured: ✅ │
|
||||
│ ├─ /api/context/ │ │ Status: DEPRECATED │
|
||||
│ │ timeline │ │ │
|
||||
│ ├─ /api/timeline/ │ │ │
|
||||
│ │ by-query │ │ │
|
||||
│ └─ /api/search/help │ │ │
|
||||
│ │ │ │
|
||||
│ Built: ✅ │ │ │
|
||||
│ Used: ✅ │ │ │
|
||||
│ Status: ACTIVE │ │ │
|
||||
└──────────────────────────┘ └──────────────────────────┘
|
||||
│ │
|
||||
└─────────┬─────────────────┘
|
||||
▼
|
||||
┌────────────────────────────────┐
|
||||
│ SessionSearch (Shared Layer) │
|
||||
│ - FTS5 queries │
|
||||
│ - SQLite operations │
|
||||
│ - Common data access │
|
||||
└────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────┐
|
||||
│ SQLite Database │
|
||||
│ ~/.claude-mem/claude-mem.db │
|
||||
└────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## HTTP Endpoints Architecture
|
||||
|
||||
### Location
|
||||
|
||||
`src/services/worker-service.ts` (lines 108-118, 748-1174)
|
||||
|
||||
### Endpoints (10 total)
|
||||
|
||||
| Endpoint | Method | Purpose | Used By |
|
||||
| -------------------------- | ------ | ----------------------------------- | ---------------- |
|
||||
| `/api/search/observations` | GET | Full-text search observations | mem-search skill |
|
||||
| `/api/search/sessions` | GET | Full-text search session summaries | mem-search skill |
|
||||
| `/api/search/prompts` | GET | Full-text search user prompts | mem-search skill |
|
||||
| `/api/search/by-concept` | GET | Find observations by concept tag | mem-search skill |
|
||||
| `/api/search/by-file` | GET | Find work related to specific files | mem-search skill |
|
||||
| `/api/search/by-type` | GET | Find observations by type | mem-search skill |
|
||||
| `/api/context/recent` | GET | Get recent session context | mem-search skill |
|
||||
| `/api/context/timeline` | GET | Get timeline around point in time | mem-search skill |
|
||||
| `/api/timeline/by-query` | GET | Search + timeline in one call | mem-search skill |
|
||||
| `/api/search/help` | GET | API documentation | mem-search skill |
|
||||
|
||||
### Implementation Pattern
|
||||
|
||||
**Example: Search Observations**
|
||||
|
||||
```typescript
|
||||
// src/services/worker-service.ts:748-781
|
||||
private handleSearchObservations(req: Request, res: Response): void {
|
||||
try {
|
||||
// 1. Parse query parameters
|
||||
const query = req.query.query as string;
|
||||
const format = (req.query.format as string) || 'full';
|
||||
const limit = parseInt(req.query.limit as string, 10) || 20;
|
||||
const project = req.query.project as string | undefined;
|
||||
|
||||
// 2. Validate required parameters
|
||||
if (!query) {
|
||||
res.status(400).json({ error: 'Missing required parameter: query' });
|
||||
return;
|
||||
}
|
||||
|
||||
// 3. Call SessionSearch (shared data layer)
|
||||
const sessionSearch = this.dbManager.getSessionSearch();
|
||||
const results = sessionSearch.searchObservations(query, { limit, project });
|
||||
|
||||
// 4. Format response based on format parameter
|
||||
res.json({
|
||||
query,
|
||||
count: results.length,
|
||||
format,
|
||||
results: format === 'index' ? results.map(r => ({
|
||||
id: r.id,
|
||||
type: r.type,
|
||||
title: r.title,
|
||||
subtitle: r.subtitle,
|
||||
created_at_epoch: r.created_at_epoch,
|
||||
project: r.project,
|
||||
score: r.score
|
||||
})) : results
|
||||
});
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Search observations failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Characteristics
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ Simple HTTP GET requests (curl-friendly)
|
||||
- ✅ Standard REST API pattern
|
||||
- ✅ Easy to test and debug
|
||||
- ✅ No MCP protocol overhead
|
||||
- ✅ Works with any HTTP client
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ⚠️ Parameter parsing duplicated across 10 endpoints
|
||||
- ⚠️ Format conversion logic duplicated
|
||||
- ⚠️ Error handling pattern repeated
|
||||
|
||||
---
|
||||
|
||||
## MCP Search Server Architecture
|
||||
|
||||
### Location
|
||||
|
||||
`src/servers/search-server.ts` (1,781 lines)
|
||||
|
||||
### Status
|
||||
|
||||
- **Built:** ✅ Yes (`plugin/scripts/search-server.mjs`, 74KB)
|
||||
- **Configured:** ✅ Yes (`.mcp.json` line 3-6)
|
||||
- **Used:** ❌ No (deprecated in v5.4.0)
|
||||
- **Maintained:** ⚠️ Source kept for reference
|
||||
|
||||
### Tools (9 total)
|
||||
|
||||
| Tool Name | Purpose | Line |
|
||||
| ----------------------- | -------------------------------------- | -------- |
|
||||
| `search_observations` | Search observations with FTS5 + Chroma | 348-422 |
|
||||
| `search_sessions` | Search session summaries | 438-490 |
|
||||
| `search_user_prompts` | Search user prompts | 506-558 |
|
||||
| `find_by_concept` | Find by concept tag | 574-626 |
|
||||
| `find_by_file` | Find by file path | 642-694 |
|
||||
| `find_by_type` | Find by observation type | 710-762 |
|
||||
| `get_recent_context` | Get recent sessions | 778-830 |
|
||||
| `get_context_timeline` | Get timeline context | 846-950 |
|
||||
| `get_timeline_by_query` | Search + timeline | 966-1064 |
|
||||
|
||||
### Implementation Pattern
|
||||
|
||||
**Example: Search Observations (MCP)**
|
||||
|
||||
```typescript
|
||||
// src/servers/search-server.ts:348-422
|
||||
{
|
||||
name: 'search_observations',
|
||||
description: 'Search observations using full-text search across titles, narratives, facts, and concepts...',
|
||||
inputSchema: z.object({
|
||||
query: z.string().describe('Search query for FTS5 full-text search'),
|
||||
format: z.enum(['index', 'full']).default('index').describe('...'),
|
||||
...filterSchema.shape
|
||||
}),
|
||||
handler: async (args: any) => {
|
||||
try {
|
||||
const { query, format = 'index', ...options } = args;
|
||||
let results: ObservationSearchResult[] = [];
|
||||
|
||||
// Hybrid search: Try Chroma semantic search first, fall back to FTS5
|
||||
if (chromaClient) {
|
||||
try {
|
||||
// Step 1: Chroma semantic search (top 100)
|
||||
const chromaResults = await queryChroma(query, 100);
|
||||
|
||||
if (chromaResults.ids.length > 0) {
|
||||
// Step 2: Filter by recency (90 days)
|
||||
const ninetyDaysAgo = Date.now() - (90 * 24 * 60 * 60 * 1000);
|
||||
const recentIds = chromaResults.ids.filter((_id, idx) => {
|
||||
const meta = chromaResults.metadatas[idx];
|
||||
return meta && meta.created_at_epoch > ninetyDaysAgo;
|
||||
});
|
||||
|
||||
// Step 3: Hydrate from SQLite
|
||||
if (recentIds.length > 0) {
|
||||
const limit = options.limit || 20;
|
||||
results = store.getObservationsByIds(recentIds, { orderBy: 'date_desc', limit });
|
||||
}
|
||||
}
|
||||
} catch (chromaError: any) {
|
||||
console.error('[search-server] Chroma query failed, falling back to FTS5:', chromaError.message);
|
||||
}
|
||||
}
|
||||
|
||||
// Fall back to FTS5 if Chroma unavailable or returned no results
|
||||
if (results.length === 0) {
|
||||
results = search.searchObservations(query, options);
|
||||
}
|
||||
|
||||
// Format results
|
||||
if (format === 'index') {
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: results.map((r, i) => formatObservationIndex(r, i)).join('\n\n') + formatSearchTips()
|
||||
}]
|
||||
};
|
||||
} else {
|
||||
return {
|
||||
content: results.map(r => ({
|
||||
type: 'resource',
|
||||
resource: {
|
||||
uri: `claude-mem://observation/${r.id}`,
|
||||
mimeType: 'text/markdown',
|
||||
text: formatObservationResult(r)
|
||||
}
|
||||
}))
|
||||
};
|
||||
}
|
||||
} catch (error: any) {
|
||||
return { content: [{ type: 'text', text: `Error: ${error.message}` }] };
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Characteristics
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ MCP protocol support
|
||||
- ✅ Hybrid search (Chroma + FTS5)
|
||||
- ✅ Rich formatting (markdown, resources)
|
||||
- ✅ Comprehensive error handling
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ Not used by skill (deprecated)
|
||||
- ❌ ~2,500 token overhead for tool definitions
|
||||
- ❌ More complex than HTTP
|
||||
- ❌ Still being built despite deprecation
|
||||
|
||||
---
|
||||
|
||||
## DRY Violation Analysis
|
||||
|
||||
### Areas of Duplication
|
||||
|
||||
#### 1. **Parameter Parsing** (10 HTTP endpoints + 9 MCP tools)
|
||||
|
||||
**HTTP Endpoints:**
|
||||
|
||||
```typescript
|
||||
// Repeated in each endpoint handler
|
||||
const query = req.query.query as string;
|
||||
const format = (req.query.format as string) || "full";
|
||||
const limit = parseInt(req.query.limit as string, 10) || 20;
|
||||
const project = req.query.project as string | undefined;
|
||||
|
||||
if (!query) {
|
||||
res.status(400).json({ error: "Missing required parameter: query" });
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
**MCP Tools:**
|
||||
|
||||
```typescript
|
||||
// Repeated in each tool handler
|
||||
const { query, format = "index", ...options } = args;
|
||||
if (!query) {
|
||||
throw new Error("Missing required parameter: query");
|
||||
}
|
||||
```
|
||||
|
||||
**Violation:** Parameter parsing logic duplicated 19 times (10 + 9)
|
||||
|
||||
#### 2. **Format Conversion** (Index vs Full)
|
||||
|
||||
**HTTP Endpoints:**
|
||||
|
||||
```typescript
|
||||
results: format === "index"
|
||||
? results.map((r) => ({
|
||||
id: r.id,
|
||||
type: r.type,
|
||||
title: r.title,
|
||||
subtitle: r.subtitle,
|
||||
created_at_epoch: r.created_at_epoch,
|
||||
project: r.project,
|
||||
score: r.score,
|
||||
}))
|
||||
: results;
|
||||
```
|
||||
|
||||
**MCP Tools:**
|
||||
|
||||
```typescript
|
||||
if (format === "index") {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: results.map((r, i) => formatObservationIndex(r, i)).join("\n\n"),
|
||||
},
|
||||
],
|
||||
};
|
||||
} else {
|
||||
return {
|
||||
content: results.map((r) => ({
|
||||
type: "resource",
|
||||
resource: {
|
||||
uri: `claude-mem://observation/${r.id}`,
|
||||
mimeType: "text/markdown",
|
||||
text: formatObservationResult(r),
|
||||
},
|
||||
})),
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**Violation:** Format conversion logic duplicated with different output formats
|
||||
|
||||
#### 3. **Search Logic Duplication**
|
||||
|
||||
**HTTP Endpoints:**
|
||||
|
||||
```typescript
|
||||
const sessionSearch = this.dbManager.getSessionSearch();
|
||||
const results = sessionSearch.searchObservations(query, { limit, project });
|
||||
```
|
||||
|
||||
**MCP Tools:**
|
||||
|
||||
```typescript
|
||||
// Hybrid search with Chroma fallback
|
||||
if (chromaClient) {
|
||||
const chromaResults = await queryChroma(query, 100);
|
||||
// ... complex hybrid logic ...
|
||||
}
|
||||
if (results.length === 0) {
|
||||
results = search.searchObservations(query, options);
|
||||
}
|
||||
```
|
||||
|
||||
**Violation:** MCP has hybrid Chroma+FTS5 search, HTTP only has FTS5
|
||||
|
||||
#### 4. **Error Handling**
|
||||
|
||||
**HTTP Endpoints:**
|
||||
|
||||
```typescript
|
||||
try {
|
||||
// ... handler logic ...
|
||||
} catch (error) {
|
||||
logger.failure("WORKER", "Search observations failed", {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
```
|
||||
|
||||
**MCP Tools:**
|
||||
|
||||
```typescript
|
||||
try {
|
||||
// ... handler logic ...
|
||||
} catch (error: any) {
|
||||
return { content: [{ type: "text", text: `Error: ${error.message}` }] };
|
||||
}
|
||||
```
|
||||
|
||||
**Violation:** Different error handling patterns
|
||||
|
||||
### DRY Compliance at Data Layer ✅
|
||||
|
||||
**Good news:** Both architectures use the **same data layer**:
|
||||
|
||||
```
|
||||
HTTP Endpoints → SessionSearch → SQLite
|
||||
MCP Tools → SessionSearch → SQLite
|
||||
```
|
||||
|
||||
The `SessionSearch` class is the **single source of truth** for data access. No duplication there.
|
||||
|
||||
---
|
||||
|
||||
## Is curl the Best Approach?
|
||||
|
||||
### Current Approach: curl Commands
|
||||
|
||||
**Example from skill:**
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
|
||||
```
|
||||
|
||||
### Alternative Approaches
|
||||
|
||||
#### 1. **MCP Tools** (Deprecated)
|
||||
|
||||
**Pros:**
|
||||
|
||||
- Native Claude Code protocol
|
||||
- Rich type definitions
|
||||
- Better error handling
|
||||
- Resource formatting
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ ~2,500 token overhead per session
|
||||
- ❌ More complex to implement
|
||||
- ❌ Requires MCP server process
|
||||
- ❌ Less accessible for external tools
|
||||
|
||||
**Verdict:** MCP was deprecated for good reasons (token overhead). curl is better.
|
||||
|
||||
#### 2. **Direct Database Access** (Not feasible)
|
||||
|
||||
**Pros:**
|
||||
|
||||
- No HTTP overhead
|
||||
- No worker process needed
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ Skills can't access files directly
|
||||
- ❌ No way to execute TypeScript/SQLite from skill
|
||||
- ❌ Would require building native bindings
|
||||
|
||||
**Verdict:** Not possible with current skill architecture.
|
||||
|
||||
#### 3. **HTTP API via curl** (Current) ✅
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ Simple, standard protocol
|
||||
- ✅ Works with skill architecture
|
||||
- ✅ Easy to test (curl in terminal)
|
||||
|
||||
- ✅ Language-agnostic
|
||||
- ✅ No MCP token overhead
|
||||
- ✅ RESTful design
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ⚠️ Requires worker service running
|
||||
- ⚠️ HTTP parsing overhead (minimal)
|
||||
|
||||
**Verdict:** **Best approach given constraints.**
|
||||
|
||||
### Why curl is Optimal
|
||||
|
||||
1. **Skill Constraints:** Skills can only execute shell commands. curl is the standard HTTP client.
|
||||
2. **Token Efficiency:** No tool definitions loaded into context (~2,250 token savings).
|
||||
3. **Progressive Disclosure:** Skill loads gradually, HTTP requests are made only when needed.
|
||||
|
||||
4. **Debuggability:** Easy to test endpoints manually with curl.
|
||||
5. **Cross-platform:** curl available on all platforms.
|
||||
|
||||
---
|
||||
|
||||
### Question: "Is it routing into the search-service MCP file or is it a DRY violation?"
|
||||
|
||||
**Answer:** Both architectures exist, creating a DRY violation:
|
||||
|
||||
1. **HTTP Endpoints** (worker-service.ts) ← **Used by skill**
|
||||
2. **MCP Server** (search-server.ts) ← **Deprecated but still built**
|
||||
|
||||
### Current State
|
||||
|
||||
```
|
||||
mem-search skill → HTTP API (worker-service.ts) → SessionSearch → SQLite
|
||||
↑
|
||||
MCP search server (deprecated) → SessionSearch ──────────────────────┘
|
||||
```
|
||||
|
||||
Both use the same data layer (SessionSearch), but:
|
||||
|
||||
- ❌ Parameter parsing duplicated
|
||||
- ❌ Format conversion duplicated
|
||||
- ❌ MCP has hybrid Chroma search, HTTP doesn't
|
||||
- ❌ MCP still being built despite deprecation
|
||||
|
||||
**You said:** "We are intentionally exposing API search endpoints
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
|
||||
│ - Web UI │
|
||||
│ - Mobile app │
|
||||
│ - VS Code extension │
|
||||
│ - CLI tools │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ HTTP API
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Worker Service HTTP API │
|
||||
│ localhost:37777/api/search/* │
|
||||
│ │
|
||||
│ - Standard REST endpoints │
|
||||
│ - JSON responses │
|
||||
│ - Query parameter API │
|
||||
│ - format=index/full support │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ SessionSearch + ChromaSync │
|
||||
│ (Shared data layer) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- Standard REST API
|
||||
- Easy to consume from any language/platform
|
||||
- Already supports format=index/full for token efficiency
|
||||
- Well-documented in skill operation guides
|
||||
- Clean JSON responses
|
||||
|
||||
---
|
||||
|
||||
## Architectural Recommendations
|
||||
|
||||
### Immediate Actions
|
||||
|
||||
#### 1. **Remove MCP Search Server** (Reduce Maintenance Burden)
|
||||
|
||||
**Problem:**
|
||||
|
||||
- MCP server is deprecated but still being built
|
||||
- Adds 1,781 lines of maintenance burden
|
||||
- Creates confusion about which search to use
|
||||
- DRY violation with HTTP endpoints
|
||||
|
||||
**Recommendation:**
|
||||
|
||||
```bash
|
||||
# Remove from build pipeline
|
||||
# scripts/build-hooks.js - already commented out, make permanent
|
||||
|
||||
# Delete configuration
|
||||
rm plugin/.mcp.json
|
||||
|
||||
# Archive source (don't delete, keep for reference)
|
||||
git mv src/servers/search-server.ts archive/search-server.ts.archived
|
||||
|
||||
# Remove built file
|
||||
rm plugin/scripts/search-server.mjs
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
|
||||
- ✅ Reduces build time
|
||||
- ✅ Eliminates confusion
|
||||
- ✅ Reduces maintenance burden
|
||||
- ✅ Removes DRY violation
|
||||
- ⚠️ Loses hybrid Chroma search in MCP (but HTTP doesn't have it anyway)
|
||||
|
||||
#### 2. **Add Hybrid Search to HTTP Endpoints** (Feature Parity)
|
||||
|
||||
**Problem:** MCP server has Chroma hybrid search, HTTP endpoints don't
|
||||
|
||||
**Recommendation:**
|
||||
|
||||
```typescript
|
||||
// src/services/worker-service.ts
|
||||
private async handleSearchObservations(req: Request, res: Response): Promise<void> {
|
||||
try {
|
||||
const { query, format, limit, project } = this.parseSearchParams(req);
|
||||
|
||||
// Try hybrid search first if Chroma available
|
||||
let results = await this.hybridSearch(query, { limit, project });
|
||||
|
||||
// Fallback to FTS5 if Chroma unavailable
|
||||
if (results.length === 0) {
|
||||
const sessionSearch = this.dbManager.getSessionSearch();
|
||||
results = sessionSearch.searchObservations(query, { limit, project });
|
||||
}
|
||||
|
||||
res.json(this.formatSearchResponse(query, results, format));
|
||||
} catch (error) {
|
||||
this.handleSearchError(res, 'Search observations failed', error);
|
||||
}
|
||||
}
|
||||
|
||||
// Extract shared methods
|
||||
private parseSearchParams(req: Request): SearchParams { /* ... */ }
|
||||
private async hybridSearch(query: string, options: SearchOptions): Promise<any[]> { /* ... */ }
|
||||
private formatSearchResponse(query: string, results: any[], format: string): any { /* ... */ }
|
||||
private handleSearchError(res: Response, message: string, error: any): void { /* ... */ }
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
|
||||
- ✅ Adds Chroma semantic search to HTTP API
|
||||
- ✅ Makes HTTP API feature-complete
|
||||
|
||||
#### 3. **Extract Shared Search Logic** (DRY Refactoring)
|
||||
|
||||
**Problem:** 10 HTTP endpoints have duplicated parameter parsing and formatting
|
||||
|
||||
**Recommendation:**
|
||||
|
||||
```typescript
|
||||
// src/services/search/SearchController.ts (new file)
|
||||
export class SearchController {
|
||||
constructor(private sessionSearch: SessionSearch, private chromaSync: ChromaSync) {}
|
||||
|
||||
async searchObservations(params: SearchParams): Promise<SearchResponse> {
|
||||
// Shared logic for observations search
|
||||
const results = await this.hybridSearch(params);
|
||||
return this.formatResponse(results, params.format);
|
||||
}
|
||||
|
||||
async searchSessions(params: SearchParams): Promise<SearchResponse> {
|
||||
// Shared logic for sessions search
|
||||
}
|
||||
|
||||
// ... other search methods
|
||||
|
||||
private async hybridSearch(params: SearchParams): Promise<any[]> {
|
||||
// Shared hybrid search logic
|
||||
}
|
||||
|
||||
private formatResponse(results: any[], format: "index" | "full"): SearchResponse {
|
||||
// Shared formatting logic
|
||||
}
|
||||
|
||||
private parseParams(req: Request): SearchParams {
|
||||
// Shared parameter parsing
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Usage in worker-service.ts:**
|
||||
|
||||
```typescript
|
||||
private searchController: SearchController;
|
||||
|
||||
private handleSearchObservations(req: Request, res: Response): void {
|
||||
try {
|
||||
const params = this.searchController.parseParams(req);
|
||||
const response = await this.searchController.searchObservations(params);
|
||||
res.json(response);
|
||||
} catch (error) {
|
||||
this.handleSearchError(res, error);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
|
||||
- ✅ Eliminates 90% of duplication across 10 endpoints
|
||||
- ✅ Single source of truth for search logic
|
||||
- ✅ Easier to test (test controller, not HTTP layer)
|
||||
- ✅ Easier to maintain
|
||||
- ✅ Easier to add new search endpoints
|
||||
|
||||
### Long-term Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Clients │
|
||||
│ ┌──────────────┬──────────────┬──────────────────────┐ │
|
||||
|
||||
│ │ Skill │ Frontend │ (CLI, IDE plugins) │ │
|
||||
│ └──────────────┴──────────────┴──────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ HTTP API (REST)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ WorkerService (Express.js) │
|
||||
│ │
|
||||
│ Route Layer (thin) │
|
||||
│ ├─ GET /api/search/observations │
|
||||
│ ├─ GET /api/search/sessions │
|
||||
│ └─ ... (delegates to controller) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ SearchController (business logic) │
|
||||
│ │
|
||||
│ ├─ searchObservations() │
|
||||
│ ├─ searchSessions() │
|
||||
│ ├─ hybridSearch() - Chroma + FTS5 │
|
||||
│ ├─ formatResponse() - index/full conversion │
|
||||
│ └─ parseParams() - parameter validation │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
┌───────────┴───────────┐
|
||||
▼ ▼
|
||||
┌──────────────────────────┐ ┌──────────────────────────┐
|
||||
│ SessionSearch (FTS5) │ │ ChromaSync (Vectors) │
|
||||
│ - searchObservations() │ │ - queryByEmbedding() │
|
||||
│ - searchSessions() │ │ - 90-day recency filter │
|
||||
│ - searchPrompts() │ │ - Hydrate from SQLite │
|
||||
└──────────────────────────┘ └──────────────────────────┘
|
||||
│ │
|
||||
└─────────┬─────────────────┘
|
||||
▼
|
||||
┌────────────────────────────────┐
|
||||
│ SQLite Database │
|
||||
│ ~/.claude-mem/claude-mem.db │
|
||||
└────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
### Current Architecture Issues
|
||||
|
||||
1. ❌ **Dual search implementations** (HTTP + deprecated MCP)
|
||||
2. ❌ **DRY violations** across 19 search handlers
|
||||
3. ❌ **MCP server still built** despite deprecation
|
||||
4. ❌ **HTTP missing hybrid Chroma search** (MCP has it)
|
||||
5. ❌ **No shared controller layer** for search logic
|
||||
|
||||
### Is curl the Best Approach?
|
||||
|
||||
**Yes.** ✅
|
||||
|
||||
Given the constraints:
|
||||
|
||||
- Skills can only execute shell commands
|
||||
|
||||
- Token efficiency vs MCP (~2,250 token savings)
|
||||
- Standard REST pattern, easy to consume
|
||||
|
||||
curl + HTTP API is the optimal architecture.
|
||||
|
||||
### Is it Routing into search-service or DRY Violation?
|
||||
|
||||
**DRY violation.** ❌
|
||||
|
||||
Both architectures exist and duplicate logic:
|
||||
|
||||
- HTTP endpoints (worker-service.ts) ← ACTIVE
|
||||
- MCP server (search-server.ts) ← DEPRECATED but BUILT
|
||||
|
||||
They share the data layer (SessionSearch) but duplicate:
|
||||
|
||||
- Parameter parsing
|
||||
- Format conversion
|
||||
- Error handling
|
||||
- Search orchestration (MCP has Chroma, HTTP doesn't)
|
||||
|
||||
### Recommendations Priority
|
||||
|
||||
**High Priority:**
|
||||
|
||||
1. ✅ Remove MCP search server entirely (archive source)
|
||||
2. ✅ Add hybrid Chroma search to HTTP endpoints
|
||||
3. ✅ Extract SearchController for shared logic
|
||||
|
||||
**Medium Priority:**
|
||||
|
||||
5. Add API versioning (/api/v1/search/\*)
|
||||
6. Add rate limiting for external access
|
||||
|
||||
**Low Priority:** 7. OpenAPI/Swagger documentation
|
||||
|
||||
9. WebSocket support for real-time search
|
||||
|
||||
### Action Plan
|
||||
|
||||
**Phase 1: Cleanup (1 day)**
|
||||
|
||||
- Remove .mcp.json
|
||||
- Archive search-server.ts
|
||||
- Update CLAUDE.md to reflect removal
|
||||
- Update build scripts to skip MCP server
|
||||
|
||||
**Phase 2: Feature Parity (2 days)**
|
||||
|
||||
- Port hybrid Chroma search from MCP to HTTP
|
||||
- Test all 10 endpoints with hybrid search
|
||||
- Update skill documentation
|
||||
|
||||
**Phase 3: DRY Refactoring (3 days)**
|
||||
|
||||
- Create SearchController class
|
||||
- Extract shared logic (parsing, formatting, errors)
|
||||
- Refactor 10 HTTP handlers to use controller
|
||||
- Add comprehensive tests
|
||||
|
||||
- Document API for external consumption
|
||||
- Add authentication/authorization (if needed)
|
||||
- Add rate limiting
|
||||
- Create OpenAPI spec
|
||||
|
||||
---
|
||||
|
||||
## Files Referenced
|
||||
|
||||
**Active:**
|
||||
|
||||
- `src/services/worker-service.ts` - HTTP endpoints (1,338 lines)
|
||||
- `src/services/sqlite/SessionSearch.ts` - FTS5 search
|
||||
- `src/services/sync/ChromaSync.ts` - Vector search
|
||||
- `plugin/skills/mem-search/SKILL.md` - Skill using HTTP API
|
||||
|
||||
**Deprecated:**
|
||||
|
||||
- `src/servers/search-server.ts` - MCP tools (1,781 lines)
|
||||
- `plugin/.mcp.json` - MCP configuration
|
||||
- `plugin/scripts/search-server.mjs` - Built MCP server (74KB)
|
||||
|
||||
**Configuration:**
|
||||
|
||||
- `CLAUDE.md` line 314 - Deprecation notice
|
||||
- `CHANGELOG.md` line 32-52 - v5.4.0 migration
|
||||
- `scripts/build-hooks.js` - Build pipeline (MCP commented out)
|
||||
@@ -0,0 +1,160 @@
|
||||
# Skill Audit Report
|
||||
|
||||
**Date:** 2025-11-10
|
||||
**Validation:** Anthropic's official skill-creator documentation
|
||||
**Skills Audited:** mem-search, search
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The mem-search skill achieves 100% compliance across all dimensions. The search skill meets technical requirements but fails effectiveness metrics critical for auto-invocation.
|
||||
|
||||
**mem-search:** Production-ready. No changes required.
|
||||
|
||||
**search:** Requires three critical fixes before Claude reliably discovers and invokes this skill.
|
||||
|
||||
## mem-search Skill Results
|
||||
|
||||
**Status:** ✅ PASS
|
||||
**Compliance:** 100% technical, 100% effectiveness
|
||||
**Files:** 17 (202-line SKILL.md + 13 operations + 2 principles)
|
||||
|
||||
### Strengths
|
||||
|
||||
The skill demonstrates exemplary effectiveness engineering:
|
||||
|
||||
1. **Trigger Design (85% concrete)**
|
||||
- Five unique identifiers: claude-mem, PM2-managed database, cross-session memory, session summaries, observations
|
||||
- Nine scope differentiation keywords
|
||||
- Explicit boundary: "NOT in the current conversation context"
|
||||
- Minimal overlap with Claude's native capabilities
|
||||
|
||||
2. **Capability Visibility (100%)**
|
||||
- All nine operations include inline "Use when" examples
|
||||
- Decision guide reduces complexity from nine operations to five common cases
|
||||
- No navigation friction
|
||||
|
||||
3. **Structure**
|
||||
- 202 lines (60% under limit)
|
||||
- Perfect progressive disclosure with token cost documentation
|
||||
- Clean file organization: operations/ and principles/ directories
|
||||
- No content duplication
|
||||
|
||||
### Issues
|
||||
|
||||
**One false positive:** Line 152 contains backslashes in regex notation `(bugfix\|feature\|decision)`. This documents parameter syntax, not Windows paths. No action required.
|
||||
|
||||
## search Skill Results
|
||||
|
||||
**Status:** ⚠️ NEEDS IMPROVEMENT
|
||||
**Compliance:** 100% technical, 67% effectiveness
|
||||
**Files:** 13 (96-line SKILL.md + 12 operations)
|
||||
|
||||
### Critical Effectiveness Issues
|
||||
|
||||
Three failures prevent reliable auto-invocation:
|
||||
|
||||
#### Issue 1: Insufficient Scope Differentiation
|
||||
|
||||
**Problem:** Description contains only two differentiation keywords (threshold: ≥3). Claude cannot distinguish this skill from native conversation memory.
|
||||
|
||||
**Current description:**
|
||||
```text
|
||||
Search claude-mem persistent memory for past sessions, observations, bugs
|
||||
fixed, features implemented, decisions made, code changes, and previous work.
|
||||
Use when answering questions about history, finding past decisions, or
|
||||
researching previous implementations.
|
||||
```
|
||||
|
||||
**Domain overlap analysis:**
|
||||
- Claude answers natively: "What bugs did we fix?" (current conversation)
|
||||
- Claude needs skill: "What bugs did we fix last week?" (external database)
|
||||
|
||||
**Fix required:**
|
||||
|
||||
```text
|
||||
Search claude-mem's external database of past sessions, observations, and
|
||||
work from previous conversations. Accesses persistent memory stored outside
|
||||
current session context - NOT information from today's conversation. Use when
|
||||
users ask about: (1) previous sessions ("what did we do last week?"),
|
||||
(2) historical work ("bugs we fixed months ago"), (3) cross-session patterns
|
||||
("how have we approached this before?"), (4) work already stored in claude-mem
|
||||
("what's in the database about X?"). Searches FTS5 full-text index across
|
||||
typed observations (bugfix/feature/refactor/decision/discovery). For current
|
||||
session memory, use native conversation context instead.
|
||||
```
|
||||
|
||||
This adds eight differentiation keywords: "external database", "past sessions", "previous conversations", "outside current session", "NOT information from today's", "last week", "months ago", "already stored in claude-mem".
|
||||
|
||||
#### Issue 2: Weak Trigger Specificity
|
||||
|
||||
**Problem:** Only 44% concrete triggers (threshold: >50%). Only one unique identifier (threshold: ≥2).
|
||||
|
||||
**Abstract triggers (low specificity):**
|
||||
- "history" (could mean git history, browser history)
|
||||
- "past work" (could mean files, commits, documents)
|
||||
- "decisions" (could mean any decision tracking)
|
||||
- "previous work" (could mean current session earlier)
|
||||
- "implementations" (could mean code in current conversation)
|
||||
|
||||
**Concrete triggers (high specificity):**
|
||||
- "claude-mem" (unique system name)
|
||||
- "persistent memory" (system-specific)
|
||||
- "sessions" (cross-session concept)
|
||||
- "observations" (system-specific)
|
||||
|
||||
**Concrete ratio:** 4/9 = 44% (fails 50% threshold)
|
||||
|
||||
**Fix required:** Add system-specific terminology: "HTTP API", "port 37777", "FTS5 full-text index", "typed observations". See combined description in Issue 1 fix.
|
||||
|
||||
#### Issue 3: Wasted Content in Body
|
||||
|
||||
**Problem:** Lines 10-22 contain "When to Use This Skill" section in SKILL.md body. This loads AFTER triggering, wastes ~200 tokens, provides no value.
|
||||
|
||||
**Reference:** [Anthropic's skill-creator documentation](https://github.com/anthropics/anthropic-quickstarts/tree/main/skill-creator) states: "The body is only loaded after triggering, so 'When to Use This Skill' sections in the body are not helpful to Claude."
|
||||
|
||||
**Fix required:** Delete lines 10-22 entirely. Move triggering examples to description field (already included in Issue 1 fix).
|
||||
|
||||
### Strengths
|
||||
|
||||
The skill demonstrates strong structure:
|
||||
|
||||
- Excellent progressive disclosure (96-line navigation hub)
|
||||
- Strong decision guide (reduces 10 operations to common cases)
|
||||
- 100% capability visibility (all operations show purpose inline)
|
||||
- No forbidden files or content duplication
|
||||
- Clean operations/ directory structure
|
||||
|
||||
### Warning
|
||||
|
||||
**Minor:** Description uses imperative "Use when" instead of third person. Change to "Useful for" or "Invoked when" for consistency with skill-creator best practices.
|
||||
|
||||
## Comparison
|
||||
|
||||
| Metric | mem-search | search | Impact |
|
||||
|--------|-----------|---------|--------|
|
||||
| Concrete triggers | 85% | 44% | search harder to discover |
|
||||
| Unique identifiers | 5+ | 1 | search less distinct |
|
||||
| Scope differentiation | 9 keywords | 2 keywords | **search conflicts with native memory** |
|
||||
| Body optimization | Clean | Wasted section | search wastes tokens |
|
||||
| Overall effectiveness | 100% | 67% | search needs fixes |
|
||||
|
||||
## Critical Recommendations
|
||||
|
||||
The search skill requires three changes before production use:
|
||||
|
||||
1. **Rewrite description** to add scope differentiation and concrete triggers (see Issue 1 fix)
|
||||
2. **Delete lines 10-22** from SKILL.md body
|
||||
3. **Convert to third person** - change "Use when" to "Useful for"
|
||||
|
||||
**Why this matters:** Without scope differentiation, Claude assumes "What bugs did we fix?" refers to current conversation, not the external claude-mem database. This causes systematic under-invocation.
|
||||
|
||||
## Reference Implementation
|
||||
|
||||
The mem-search skill serves as a reference implementation for:
|
||||
|
||||
- Trigger design with explicit scope boundaries
|
||||
- Progressive disclosure with token efficiency documentation
|
||||
- Inline capability visibility eliminating navigation friction
|
||||
- Decision guides reducing cognitive load
|
||||
|
||||
Study mem-search when creating skills that overlap with Claude's native capabilities.
|
||||
@@ -0,0 +1,467 @@
|
||||
# Transcript: 57dcc12f-4751-46bb-82b4-2aa96a3e226d.jsonl
|
||||
|
||||
**Generated:** 11/13/2025, 3:58:53 PM
|
||||
**Total Entries:** 448
|
||||
**Entry Types:** {
|
||||
"summary": 149,
|
||||
"file-history-snapshot": 20,
|
||||
"user": 98,
|
||||
"assistant": 181
|
||||
}
|
||||
**Showing:** First 10 conversation turns
|
||||
|
||||
---
|
||||
|
||||
## Turn 1
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:09:47.231Z
|
||||
**UUID:** 5c4b246c-2100-4e6b-a275-f0bb62bba020
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
I have a question regarding the context timeline that we made. and it's a bit of a thinker...
|
||||
|
||||
when i first started out working on claude-mem my idea was using a fast index of memories with links to access them on demand. I thought that claude-mem would be searching for all the things it needed to get the job done, but what I found is that the description of the work done + the files touched actually improved performance tremendously, and without actively searching for things constantly.
|
||||
|
||||
I just had a thought that maybe claude isn't searching, because when it reads what it did, it already "knows" how it did that, for the most part. Would you say I'm on the right track? I want you to ultrathink about WHY claude-mem's session start context works so well, from your OWN perspective
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:10:28.576Z
|
||||
**UUID:** 1dd9591b-c98b-4031-ac0a-2f7e067b427a
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> This is a fascinating question about the fundamental mechanics of how LLMs process context and why claude-mem's approach is working so well. The user is asking me to think deeply about WHY the session start context (descriptions + file paths) works better than active searching.
|
||||
>
|
||||
> Let me think about this from my own perspective as Claude...
|
||||
>
|
||||
> The user's hypothesis is: "Claude isn't searching because when it reads what it did, it already 'knows' how it did that."
|
||||
>
|
||||
> This is a really interesting observation. Let me think about what happens when I see context at session start vs when I actively search:
|
||||
>
|
||||
> 1. **Session Start Context**: I see summaries like "Fixed authentication bug in src/auth/login.ts by updating token validation logic" with file paths
|
||||
> 2. **Active Search**: I use grep/search tools to find specific code or patterns
|
||||
>
|
||||
> The key insight the user is pointing to: When I see the session summary, it's triggering my training knowledge about how these kinds of problems are typically solved. The description + file path combo is enough to reconstruct the likely solution pattern.
|
||||
>
|
||||
> Let me ultrathink about this...
|
||||
>
|
||||
> **Information Density & Compression**:
|
||||
> - A summary like "Added JWT token refresh logic to src/auth/token-manager.ts" is highly compressed but information-rich
|
||||
> - It contains: the pattern (JWT refresh), the location (token-manager), and the action type (added)
|
||||
> - This maps to my training data about how JWT refresh is typically implemented
|
||||
> - The file path gives architectural context about where this lives in the codebase
|
||||
>
|
||||
> **Pattern Recognition vs Literal Recall**:
|
||||
> - I don't need to see the exact code to "know" what was probably done
|
||||
> - JWT token refresh follows common patterns that I've seen thousands of times in training
|
||||
> - The summary activates these patterns in my understanding
|
||||
> - It's like how a human developer reading "fixed the N+1 query problem" immediately knows the general shape of the solution without seeing the code
|
||||
>
|
||||
> **Contextual Scaffolding**:
|
||||
> - File paths provide architectural scaffolding: "src/hooks/context-hook.ts" tells me this is a hook in a hooks directory
|
||||
>
|
||||
>
|
||||
> - The summary provides semantic scaffolding: what was accomplished
|
||||
> - Together they form a mental model of the codebase without needing to read every line
|
||||
>
|
||||
> **Search is for Unknowns, Summaries are for Knowns**:
|
||||
> - I search when I don't know something: "Where is error handling done?"
|
||||
> - I don't need to search when the summary tells me: "Enhanced error handling in src/services/worker-service.ts"
|
||||
> - The summary has already done the work of locating and describing the relevant information
|
||||
>
|
||||
> **The Compression-Decompression Cycle**:
|
||||
> - The AI compression (creating observations) is like creating a highly optimized index
|
||||
> - When I encounter a summary, I'm essentially decompressing that index into actionable knowledge
|
||||
> - This process is far more efficient than searching through raw code or logs
|
||||
>
|
||||
> The summaries act like cognitive shortcuts, instantly reconstructing complex contextual understanding through minimal, strategic information.
|
||||
>
|
||||
> **Why This Works Better Than Constant Searching**:
|
||||
> 1. Drastically reduces mental overhead of navigating complex systems
|
||||
> 2. Enables rapid pattern recognition across different code segments
|
||||
> 3. Provides architectural insights without deep diving into implementation details
|
||||
> 4. Minimizes computational and cognitive resources spent on exploration
|
||||
> 5. Delivers pre-filtered, contextually relevant information
|
||||
>
|
||||
> **The "Already Knows" Hypothesis**:
|
||||
> I recognize complex technical changes instantly through pattern matching. When encountering a concise code modification description, my training allows immediate comprehension by activating relevant technical knowledge domains. The summary triggers an intuitive understanding of systemic implications, similar to how experienced developers quickly grasp architectural shifts through minimal context.
|
||||
>
|
||||
> **The Real Magic**:
|
||||
> Leveraging deep pattern recognition capabilities, I can reconstruct technical landscapes from minimal descriptive signals. File paths and succinct change descriptions become powerful activation keys that unlock comprehensive understanding without exhaustive code examination.
|
||||
>
|
||||
> **Why Links-On-Demand Falls Short**:
|
||||
> Retrieving information through multiple steps disrupts cognitive flow. Contextual understanding emerges more naturally through continuous narrative, where each detail builds upon previous insights. Searching and parsing interrupt the organic comprehension process, making knowledge integration less fluid and intuitive.
|
||||
>
|
||||
> **The Timeline Innovation**:
|
||||
> Project evolution unfolds like a narrative, with each session representing a chapter in technological development. Temporal relationships between changes reveal deeper systemic transformations, mirroring how humans conceptualize complex system growth through interconnected stories of incremental progress.
|
||||
>
|
||||
> The essence lies in providing compressed, meaningful representations that leverage my inherent training knowledge. By presenting contextual frameworks that enable rapid pattern recognition, understanding emerges more elegantly than through fragmented, manually retrieved information.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 10
|
||||
- Output: 1495
|
||||
- Cache creation: 28036
|
||||
|
||||
---
|
||||
|
||||
## Turn 2
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:10:31.963Z
|
||||
**UUID:** 364676a7-51c3-4036-afc3-7ff8f7301a8f
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_01T477WUra1sDR6gHaqZHhKT`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 1,
|
||||
"totalThoughts": 8,
|
||||
"nextThoughtNeeded": true,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 1
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:10:35.939Z
|
||||
**UUID:** 201d5077-aae8-4e26-9c5e-6a63f1ca88fa
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> Good, I'm using the sequential thinking tool to really dig into this. Let me continue with the next thought about information density.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 12
|
||||
- Output: 256
|
||||
- Cache creation: 1529
|
||||
- Cache read: 28036
|
||||
|
||||
---
|
||||
|
||||
## Turn 3
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:10:38.354Z
|
||||
**UUID:** 6c2f5da2-6a5a-4742-a46d-6dd5d193ad39
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_01QMMWEJm8yH25VtBmAzYbmy`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 2,
|
||||
"totalThoughts": 8,
|
||||
"nextThoughtNeeded": true,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 2
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:10:41.841Z
|
||||
**UUID:** 3e6f3e78-5c12-4739-a7aa-a8c6fc9d2a33
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> Good, now let me think about the difference between pattern recognition and literal recall.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 11
|
||||
- Output: 262
|
||||
- Cache creation: 323
|
||||
- Cache read: 29565
|
||||
|
||||
---
|
||||
|
||||
## Turn 4
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:10:45.430Z
|
||||
**UUID:** f99751a0-b13a-446a-9b3b-dfc08ffb8e5b
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_01JwNsDV2HcPnDE2eFJdG7zF`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 3,
|
||||
"totalThoughts": 8,
|
||||
"nextThoughtNeeded": true,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 3
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:10:48.954Z
|
||||
**UUID:** 11c28930-0797-4f9e-ada8-5eaa27c70b47
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> Now let me think about the architectural scaffolding aspect.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 10
|
||||
- Output: 260
|
||||
- Cache creation: 329
|
||||
- Cache read: 29888
|
||||
|
||||
---
|
||||
|
||||
## Turn 5
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:10:53.302Z
|
||||
**UUID:** 6f013ecd-a0f0-4bb9-8eca-e6e8d3f80a53
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_016nFXyGw4SsdKfhUb9eBtkv`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 4,
|
||||
"totalThoughts": 8,
|
||||
"nextThoughtNeeded": true,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 4
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:10:56.583Z
|
||||
**UUID:** c6d275b3-7f28-4bdd-8a81-80f1a73ffbc7
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> Now let me think about why this works better than active searching.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 9
|
||||
- Output: 258
|
||||
- Cache creation: 327
|
||||
- Cache read: 30217
|
||||
|
||||
---
|
||||
|
||||
## Turn 6
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:11:00.306Z
|
||||
**UUID:** 885fbc9f-e767-4a16-a8aa-b8bb54849d67
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_013QydhPuiBKVRHh1vbD8eL8`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 5,
|
||||
"totalThoughts": 8,
|
||||
"nextThoughtNeeded": true,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 5
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:11:04.430Z
|
||||
**UUID:** 780354e4-abbf-4d97-b1e2-4f3abc5aef8f
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> Now let me think about the compression-decompression cycle - this is a key insight.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 8
|
||||
- Output: 264
|
||||
- Cache creation: 325
|
||||
- Cache read: 30544
|
||||
|
||||
---
|
||||
|
||||
## Turn 7
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:11:07.748Z
|
||||
**UUID:** 462efec2-17c3-47f7-9171-622e00f6bbf7
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_01Wa5zzkzrYqf5Cfb5UZi5H6`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 6,
|
||||
"totalThoughts": 8,
|
||||
"nextThoughtNeeded": true,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 6
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:11:12.931Z
|
||||
**UUID:** 325a8179-8291-4198-a661-eae8f02d57b2
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> The user wants me to continue with the sequential thinking about why claude-mem's session start context works so well. I should continue the thinking process and not mention the todo reminder since this is a theoretical discussion, not a task execution.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 10
|
||||
- Output: 328
|
||||
- Cache creation: 1883
|
||||
- Cache read: 28036
|
||||
|
||||
---
|
||||
|
||||
## Turn 8
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:11:17.712Z
|
||||
**UUID:** 189d903f-a901-465f-bd6b-8addf44cd5f5
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_015VKanT9FxnCSsZiTduHd2y`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 7,
|
||||
"totalThoughts": 9,
|
||||
"nextThoughtNeeded": true,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 7
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:11:21.820Z
|
||||
**UUID:** 96ca3f03-f889-454d-a13a-96e7a36a3daa
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> I want to add one more thought about the temporal/chronological aspect and how that creates a narrative structure that aids understanding.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 12
|
||||
- Output: 290
|
||||
- Cache creation: 392
|
||||
- Cache read: 29919
|
||||
|
||||
---
|
||||
|
||||
## Turn 9
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:11:26.591Z
|
||||
**UUID:** cfd56a68-8628-4811-9df5-75a8e557bec6
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_01EWtaz38nageXXEjYzzm2hu`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 8,
|
||||
"totalThoughts": 9,
|
||||
"nextThoughtNeeded": true,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 8
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:11:30.749Z
|
||||
**UUID:** bb106bb5-9696-4538-b810-d9fa56bcffff
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** tool_use
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> Now for the synthesis - bringing all these threads together to answer the user's question about WHY it works so well from my own perspective.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 11
|
||||
- Output: 360
|
||||
- Cache creation: 357
|
||||
- Cache read: 30311
|
||||
|
||||
---
|
||||
|
||||
## Turn 10
|
||||
|
||||
### 👤 User
|
||||
**Timestamp:** 2025-11-13T17:11:36.692Z
|
||||
**UUID:** 0aa4f55d-9f5a-485f-9aef-dd90f9e80391
|
||||
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
|
||||
**CWD:** /Users/alexnewman/Scripts/claude-mem
|
||||
|
||||
**Tool Results Submitted (1):**
|
||||
|
||||
- **Tool Use ID:** `toolu_01R5Sy5zet7JCPmXhvDrkMAR`
|
||||
```json
|
||||
{
|
||||
"thoughtNumber": 9,
|
||||
"totalThoughts": 9,
|
||||
"nextThoughtNeeded": false,
|
||||
"branches": [],
|
||||
"thoughtHistoryLength": 9
|
||||
}
|
||||
```
|
||||
|
||||
### 🤖 Assistant
|
||||
**Timestamp:** 2025-11-13T17:11:43.435Z
|
||||
**UUID:** 2a5c5c37-9124-441f-ac1f-3f9bff08281c
|
||||
**Model:** claude-sonnet-4-5-20250929
|
||||
**Stop Reason:** end_turn
|
||||
|
||||
**💭 Thinking:**
|
||||
|
||||
> This is a fascinating philosophical question about how I process information. The user is asking me to think deeply about WHY claude-mem's session start context works so well from MY perspective as an LLM. They've noticed something important: I don't seem to need to actively search when I have good summaries, and they're hypothesizing it's because the summaries trigger pattern recognition rather than requiring literal recall.
|
||||
>
|
||||
> This is not a task-oriented request - it's a meta-cognitive question about how I think. I should not use TodoWrite here as this is a philosophical discussion, not a task to track.
|
||||
>
|
||||
> The sequential thinking tool has helped me organize my thoughts. Let me now synthesize this into a clear response about my actual experience.
|
||||
|
||||
**📊 Token Usage:**
|
||||
- Input: 10
|
||||
- Output: 936
|
||||
- Cache creation: 1140
|
||||
- Cache read: 29919
|
||||
|
||||
---
|
||||
|
||||
|
||||
*... 87 more turns not shown*
|
||||
@@ -0,0 +1,632 @@
|
||||
# Transcript Data Analysis: Available Context for Memory Worker
|
||||
|
||||
**Generated:** 2025-11-13
|
||||
**Purpose:** Document what contextual data exists in Claude Code transcripts and identify opportunities to improve memory worker observation generation.
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**Current State:** The memory worker receives isolated tool executions via `save-hook.ts`:
|
||||
- Tool name
|
||||
- Tool input (parameters)
|
||||
- Tool output (results)
|
||||
|
||||
**Available in Transcripts:** Rich contextual data that could dramatically improve observation quality:
|
||||
- User's original request/intent
|
||||
- Assistant's reasoning (thinking blocks)
|
||||
- Full conversation context
|
||||
- Tool result data
|
||||
- Token usage and performance metrics
|
||||
- Session metadata (timestamps, UUIDs, CWD)
|
||||
|
||||
**Recommendation:** Enhance the memory worker to receive full conversation context for each tool execution, not just isolated tool data.
|
||||
|
||||
---
|
||||
|
||||
## Transcript Structure
|
||||
|
||||
### Entry Types
|
||||
|
||||
The transcript file (`~/.claude/projects/-{project}/session-id.jsonl`) contains:
|
||||
|
||||
```
|
||||
- summary entries (149 in sample)
|
||||
- file-history-snapshot entries (18 in sample)
|
||||
- user entries (86 in sample)
|
||||
- assistant entries (155 in sample)
|
||||
```
|
||||
|
||||
### Conversation Turn Pattern
|
||||
|
||||
Each conversation turn consists of:
|
||||
1. **User Entry** - User's request
|
||||
2. **Assistant Entry** - Assistant's response
|
||||
3. **User Entry** - Tool results submitted back (automatic)
|
||||
4. **Assistant Entry** - Assistant processes results and continues
|
||||
|
||||
This creates a pattern: User → Assistant → User (tool results) → Assistant (continues) → ...
|
||||
|
||||
---
|
||||
|
||||
## Available Data by Entry Type
|
||||
|
||||
### 1. User Entries
|
||||
|
||||
**Current Save-Hook Access:**
|
||||
- Tool name
|
||||
- Tool input
|
||||
- Tool output
|
||||
|
||||
**Additional Data Available in User Entries:**
|
||||
|
||||
```typescript
|
||||
interface UserTranscriptEntry {
|
||||
type: 'user';
|
||||
timestamp: string; // ISO timestamp
|
||||
uuid: string; // Unique entry ID
|
||||
sessionId: string; // Session identifier
|
||||
cwd: string; // Working directory
|
||||
parentUuid?: string; // Parent entry reference
|
||||
isSidechain: boolean; // Is this a side conversation?
|
||||
userType: string; // 'human' or 'system'
|
||||
version: string; // Claude Code version
|
||||
|
||||
message: {
|
||||
role: 'user';
|
||||
content: string | ContentItem[]; // Can be text or structured
|
||||
};
|
||||
|
||||
toolUseResult?: ToolUseResult; // Legacy field, may contain results
|
||||
}
|
||||
```
|
||||
|
||||
**When `content` is an array, it contains:**
|
||||
- Text blocks with user's actual request
|
||||
- Tool result blocks with complete output data
|
||||
|
||||
**Example Structure:**
|
||||
```json
|
||||
{
|
||||
"type": "user",
|
||||
"timestamp": "2025-11-13T17:10:31.963Z",
|
||||
"uuid": "364676a7-51c3-4036-afc3-7ff8f7301a8f",
|
||||
"sessionId": "57dcc12f-4751-46bb-82b4-2aa96a3e226d",
|
||||
"cwd": "/Users/alexnewman/Scripts/claude-mem",
|
||||
"message": {
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": "toolu_01T477WUra1sDR6gHaqZHhKT",
|
||||
"content": "[actual tool output data]"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Assistant Entries
|
||||
|
||||
**Current Save-Hook Access:**
|
||||
- Nothing from assistant entries (they happen after tool execution)
|
||||
|
||||
**Available Data in Assistant Entries:**
|
||||
|
||||
```typescript
|
||||
interface AssistantTranscriptEntry {
|
||||
type: 'assistant';
|
||||
timestamp: string;
|
||||
uuid: string;
|
||||
sessionId: string;
|
||||
cwd: string;
|
||||
parentUuid?: string;
|
||||
isSidechain: boolean;
|
||||
userType: string;
|
||||
version: string;
|
||||
requestId?: string; // API request ID
|
||||
|
||||
message: {
|
||||
id: string;
|
||||
type: 'message';
|
||||
role: 'assistant';
|
||||
model: string; // e.g., "claude-sonnet-4-5-20250929"
|
||||
content: ContentItem[]; // Array of content blocks
|
||||
stop_reason?: string; // 'tool_use' | 'end_turn' | etc.
|
||||
stop_sequence?: string;
|
||||
usage?: UsageInfo; // Token usage stats
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**Content Block Types in `message.content`:**
|
||||
|
||||
1. **Thinking Blocks** - Internal reasoning before acting
|
||||
```typescript
|
||||
{
|
||||
type: 'thinking';
|
||||
thinking: string; // Full reasoning text
|
||||
signature?: string;
|
||||
}
|
||||
```
|
||||
|
||||
2. **Text Blocks** - Assistant's visible response
|
||||
```typescript
|
||||
{
|
||||
type: 'text';
|
||||
text: string; // Response text
|
||||
}
|
||||
```
|
||||
|
||||
3. **Tool Use Blocks** - Tool invocations
|
||||
```typescript
|
||||
{
|
||||
type: 'tool_use';
|
||||
id: string; // Tool use ID
|
||||
name: string; // Tool name (e.g., 'Read', 'Edit')
|
||||
input: Record<string, any>; // Complete tool parameters
|
||||
}
|
||||
```
|
||||
|
||||
**Token Usage Data:**
|
||||
```typescript
|
||||
interface UsageInfo {
|
||||
input_tokens?: number;
|
||||
output_tokens?: number;
|
||||
cache_creation_input_tokens?: number;
|
||||
cache_read_input_tokens?: number;
|
||||
service_tier?: string;
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Summary Entries
|
||||
|
||||
```typescript
|
||||
interface SummaryTranscriptEntry {
|
||||
type: 'summary';
|
||||
summary: string; // Generated summary text
|
||||
leafUuid: string; // UUID of summarized entry
|
||||
cwd?: string;
|
||||
}
|
||||
```
|
||||
|
||||
These appear frequently (149 in sample) and provide high-level summaries of work done.
|
||||
|
||||
---
|
||||
|
||||
## Data Flow: Current vs Potential
|
||||
|
||||
### Current Flow (Save-Hook Only)
|
||||
|
||||
```
|
||||
User: "Fix the bug in login.ts"
|
||||
↓
|
||||
Assistant: [uses Edit tool]
|
||||
↓
|
||||
Tool Execution: Edit(file_path: "login.ts", old_string: "...", new_string: "...")
|
||||
↓
|
||||
Save-Hook receives:
|
||||
- toolName: "Edit"
|
||||
- toolInput: { file_path: "login.ts", old_string: "...", new_string: "..." }
|
||||
- toolOutput: { success: true }
|
||||
↓
|
||||
Memory Worker generates observation from ONLY tool data
|
||||
- No user intent
|
||||
- No assistant reasoning
|
||||
- No context about WHY this change was made
|
||||
```
|
||||
|
||||
### Enhanced Flow (With Transcript Context)
|
||||
|
||||
```
|
||||
User: "Fix the authentication bug - users getting logged out randomly"
|
||||
↓
|
||||
Assistant (thinking): "This sounds like a token expiration issue.
|
||||
Let me check the JWT handling in login.ts..."
|
||||
↓
|
||||
Assistant (uses Edit tool)
|
||||
↓
|
||||
Save-Hook receives:
|
||||
- toolName: "Edit"
|
||||
- toolInput: { file_path: "login.ts", ... }
|
||||
- toolOutput: { success: true }
|
||||
- PLUS:
|
||||
- userRequest: "Fix the authentication bug - users getting logged out randomly"
|
||||
- assistantReasoning: "This sounds like a token expiration issue..."
|
||||
- conversationContext: Previous 2-3 turns
|
||||
- sessionMetadata: { cwd, timestamp, sessionId }
|
||||
↓
|
||||
Memory Worker generates richer observation:
|
||||
- "Fixed authentication bug causing random logouts"
|
||||
- "Problem: JWT tokens expiring too quickly"
|
||||
- "Solution: Updated token expiration to 24h in login.ts"
|
||||
- "Files: src/auth/login.ts"
|
||||
- "Concepts: authentication, token-management, bugfix"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Specific Opportunities
|
||||
|
||||
### 1. User Intent Extraction
|
||||
|
||||
**Problem:** Current observations lack user intent.
|
||||
|
||||
**Solution:** Parse the most recent user text entry before the tool execution.
|
||||
|
||||
**Implementation:**
|
||||
- Walk backward from tool execution entry
|
||||
- Find first user entry with text content
|
||||
- Extract text blocks (filter out tool_result blocks)
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
// In save-hook.ts
|
||||
const userEntries = parser.getUserEntries();
|
||||
const recentUserMessage = findUserMessageBeforeTool(userEntries, toolExecutionTimestamp);
|
||||
const userIntent = extractTextFromContent(recentUserMessage.content);
|
||||
```
|
||||
|
||||
### 2. Assistant Reasoning
|
||||
|
||||
**Problem:** We don't capture WHY the assistant chose to use a tool.
|
||||
|
||||
**Solution:** Extract thinking blocks from assistant entry immediately before tool use.
|
||||
|
||||
**Implementation:**
|
||||
- Find assistant entry that contains the tool_use block
|
||||
- Extract thinking blocks from same entry
|
||||
- Include first ~500 chars of thinking in observation context
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const assistantEntry = findAssistantEntryWithToolUse(toolUseId);
|
||||
const thinkingBlocks = assistantEntry.message.content.filter(c => c.type === 'thinking');
|
||||
const reasoning = thinkingBlocks.map(b => b.thinking).join('\n');
|
||||
```
|
||||
|
||||
### 3. Tool Results Context
|
||||
|
||||
**Problem:** Tool output alone doesn't show what was found or changed.
|
||||
|
||||
**Solution:** Access full tool result content from next user entry.
|
||||
|
||||
**Implementation:**
|
||||
- Tool execution happens in assistant entry
|
||||
- Results come back in next user entry as tool_result content
|
||||
- Save-hook can access both
|
||||
|
||||
**Current Structure:**
|
||||
```
|
||||
Assistant Entry:
|
||||
{ type: 'tool_use', id: 'toolu_123', name: 'Read', input: {...} }
|
||||
↓
|
||||
User Entry (automatic):
|
||||
{ type: 'tool_result', tool_use_id: 'toolu_123', content: "file contents..." }
|
||||
```
|
||||
|
||||
**Opportunity:** Match tool_use_id to tool_result and include full result content.
|
||||
|
||||
### 4. Conversation Context
|
||||
|
||||
**Problem:** Isolated tool executions miss the larger conversation flow.
|
||||
|
||||
**Solution:** Include last N conversation turns (2-3 turns is usually sufficient).
|
||||
|
||||
**Implementation:**
|
||||
- Get entries from transcript within time window (e.g., last 5 minutes)
|
||||
- Include user messages and assistant text responses
|
||||
- Exclude thinking blocks to save tokens
|
||||
|
||||
**Example Context:**
|
||||
```
|
||||
Turn 1:
|
||||
User: "I need to add dark mode support"
|
||||
Assistant: "I'll help you add dark mode. Let me start by..."
|
||||
|
||||
Turn 2:
|
||||
User: [tool results]
|
||||
Assistant: "Now I'll update the theme configuration..."
|
||||
|
||||
Turn 3: [current tool execution]
|
||||
```
|
||||
|
||||
### 5. Session Metadata
|
||||
|
||||
**Problem:** Observations lack temporal and project context.
|
||||
|
||||
**Solution:** Include session metadata in observation generation.
|
||||
|
||||
**Available Fields:**
|
||||
- `cwd` - Working directory (project path)
|
||||
- `timestamp` - Exact time of execution
|
||||
- `sessionId` - Session identifier
|
||||
- `uuid` - Entry identifier
|
||||
- `version` - Claude Code version
|
||||
|
||||
**Use Case:** Helps with project-specific context and temporal queries.
|
||||
|
||||
### 6. Token Usage Metrics
|
||||
|
||||
**Problem:** No visibility into performance and cost.
|
||||
|
||||
**Solution:** Track token usage per observation.
|
||||
|
||||
**Available Data:**
|
||||
- Input tokens
|
||||
- Output tokens
|
||||
- Cache creation tokens
|
||||
- Cache read tokens
|
||||
|
||||
**Use Case:**
|
||||
- Performance monitoring
|
||||
- Cost attribution
|
||||
- Cache effectiveness analysis
|
||||
|
||||
---
|
||||
|
||||
## Recommended Implementation Strategy
|
||||
|
||||
### Phase 1: User Intent (High Impact, Low Effort)
|
||||
|
||||
**Change:** Modify save-hook to extract user's most recent message.
|
||||
|
||||
**Implementation:**
|
||||
```typescript
|
||||
// In save-hook.ts
|
||||
import { TranscriptParser } from '../utils/transcript-parser';
|
||||
|
||||
const parser = new TranscriptParser(transcriptPath);
|
||||
const userIntent = parser.getLastUserMessage();
|
||||
|
||||
// Send to worker
|
||||
await workerService.saveToolExecution({
|
||||
...existingData,
|
||||
userIntent, // NEW
|
||||
});
|
||||
```
|
||||
|
||||
**Impact:** Observations now include "what the user wanted to do".
|
||||
|
||||
### Phase 2: Assistant Reasoning (High Impact, Medium Effort)
|
||||
|
||||
**Change:** Extract thinking blocks from assistant entry containing tool use.
|
||||
|
||||
**Implementation:**
|
||||
```typescript
|
||||
const assistantEntries = parser.getAssistantEntries();
|
||||
const toolUseEntry = findEntryWithToolUse(assistantEntries, toolUseId);
|
||||
const thinking = extractThinkingBlocks(toolUseEntry);
|
||||
|
||||
await workerService.saveToolExecution({
|
||||
...existingData,
|
||||
userIntent,
|
||||
assistantReasoning: thinking, // NEW
|
||||
});
|
||||
```
|
||||
|
||||
**Impact:** Observations include "why the assistant chose this approach".
|
||||
|
||||
### Phase 3: Conversation Context (Medium Impact, High Effort)
|
||||
|
||||
**Change:** Include last 2-3 conversation turns.
|
||||
|
||||
**Implementation:**
|
||||
```typescript
|
||||
const recentTurns = getRecentConversationTurns(parser, 3);
|
||||
|
||||
await workerService.saveToolExecution({
|
||||
...existingData,
|
||||
userIntent,
|
||||
assistantReasoning: thinking,
|
||||
conversationContext: recentTurns, // NEW
|
||||
});
|
||||
```
|
||||
|
||||
**Impact:** Observations understand multi-turn workflows.
|
||||
|
||||
### Phase 4: Enhanced Metadata (Low Impact, Low Effort)
|
||||
|
||||
**Change:** Include session and performance metadata.
|
||||
|
||||
**Implementation:**
|
||||
```typescript
|
||||
await workerService.saveToolExecution({
|
||||
...existingData,
|
||||
userIntent,
|
||||
assistantReasoning: thinking,
|
||||
conversationContext: recentTurns,
|
||||
metadata: { // NEW
|
||||
cwd: entry.cwd,
|
||||
timestamp: entry.timestamp,
|
||||
sessionId: entry.sessionId,
|
||||
tokenUsage: entry.message.usage,
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
**Impact:** Better analytics and debugging.
|
||||
|
||||
---
|
||||
|
||||
## Example: Before and After
|
||||
|
||||
### Current Observation (Tool Data Only)
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "feature",
|
||||
"title": "Updated login.ts",
|
||||
"narrative": "Modified authentication logic in src/auth/login.ts",
|
||||
"files": ["src/auth/login.ts"],
|
||||
"concepts": ["authentication"],
|
||||
"facts": []
|
||||
}
|
||||
```
|
||||
|
||||
### Enhanced Observation (With Transcript Context)
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "bugfix",
|
||||
"title": "Fixed authentication bug causing random logouts",
|
||||
"narrative": "Users were experiencing random logouts due to JWT token expiration. Updated token expiration from 1h to 24h in token validation logic. Modified src/auth/login.ts to use longer-lived tokens and improved error handling for expired tokens.",
|
||||
"files": ["src/auth/login.ts"],
|
||||
"concepts": ["authentication", "jwt", "token-management", "bugfix"],
|
||||
"facts": [
|
||||
"JWT token expiration was too short (1h)",
|
||||
"Updated expiration to 24h",
|
||||
"Added error handling for expired tokens"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Improvement:**
|
||||
- Clear problem statement
|
||||
- Explicit solution
|
||||
- Specific technical details
|
||||
- Better concept tagging
|
||||
- Actionable facts
|
||||
|
||||
---
|
||||
|
||||
## Technical Considerations
|
||||
|
||||
### 1. Performance
|
||||
|
||||
**Concern:** Parsing entire transcript on every tool execution.
|
||||
|
||||
**Solution:**
|
||||
- TranscriptParser already loads full file (unavoidable)
|
||||
- Use caching for transcript parsing within same session
|
||||
- Only parse once per session, reuse parsed entries
|
||||
|
||||
**Benchmark:**
|
||||
- Current: ~10ms to parse 408-line transcript
|
||||
- Impact: Negligible (save-hook already reads transcript)
|
||||
|
||||
### 2. Token Usage
|
||||
|
||||
**Concern:** Sending more context to worker increases tokens.
|
||||
|
||||
**Solution:**
|
||||
- Thinking blocks: Limit to first 500 chars
|
||||
- Conversation context: Only last 2-3 turns
|
||||
- Tool results: Truncate large outputs to 500 chars
|
||||
- User intent: Full text (usually short)
|
||||
|
||||
**Estimate:**
|
||||
- Current: ~200 tokens per observation generation
|
||||
- Enhanced: ~500 tokens per observation generation
|
||||
- Increase: ~150%
|
||||
- Cost: Still < $0.001 per observation with Haiku
|
||||
|
||||
### 3. Implementation Complexity
|
||||
|
||||
**Concern:** Matching tool executions to transcript entries.
|
||||
|
||||
**Solution:**
|
||||
- Tool use IDs are in both places
|
||||
- Timestamps provide ordering
|
||||
- UUID chains provide parent-child relationships
|
||||
|
||||
**Example Matching:**
|
||||
```typescript
|
||||
function findToolContext(parser: TranscriptParser, toolUseId: string) {
|
||||
// 1. Find assistant entry with tool_use block
|
||||
const assistantEntry = parser.getAssistantEntries()
|
||||
.find(entry =>
|
||||
entry.message.content.some(c =>
|
||||
c.type === 'tool_use' && c.id === toolUseId
|
||||
)
|
||||
);
|
||||
|
||||
// 2. Find next user entry with tool_result
|
||||
const userEntry = parser.getUserEntries()
|
||||
.find(entry =>
|
||||
entry.message.content.some(c =>
|
||||
c.type === 'tool_result' && c.tool_use_id === toolUseId
|
||||
)
|
||||
);
|
||||
|
||||
return { assistantEntry, userEntry };
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Validate Approach**
|
||||
- Review this analysis with project team
|
||||
- Confirm data availability in all transcript scenarios
|
||||
- Identify any privacy concerns
|
||||
|
||||
2. **Implement Phase 1**
|
||||
- Update save-hook.ts to extract user intent
|
||||
- Modify worker service to accept new fields
|
||||
- Update observation prompt to use user intent
|
||||
|
||||
3. **Test and Measure**
|
||||
- Compare observation quality before/after
|
||||
- Measure token usage increase
|
||||
- Validate performance impact
|
||||
|
||||
4. **Iterate**
|
||||
- Roll out Phase 2 (assistant reasoning)
|
||||
- Roll out Phase 3 (conversation context)
|
||||
- Monitor improvements at each phase
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Data Samples
|
||||
|
||||
### Complete Markdown Representation
|
||||
|
||||
See `/Users/alexnewman/Scripts/claude-mem/docs/context/transcript-complete-readable.md` for a full 1:1 markdown representation of the first 10 conversation turns from the sample transcript, including:
|
||||
- Complete user messages
|
||||
- Full assistant responses
|
||||
- Thinking blocks (truncated to 2000 chars)
|
||||
- Tool uses with complete input JSON
|
||||
- Tool results with actual output data (truncated to 500 chars)
|
||||
- Token usage stats
|
||||
- All metadata (timestamps, UUIDs, session IDs, CWD)
|
||||
|
||||
### Sample Tool Result Structure
|
||||
|
||||
```typescript
|
||||
// User entry containing tool result
|
||||
{
|
||||
"type": "user",
|
||||
"message": {
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": "toolu_01T477WUra1sDR6gHaqZHhKT",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "{\n \"thoughtNumber\": 1,\n \"totalThoughts\": 8,\n \"nextThoughtNeeded\": true,\n \"branches\": [],\n \"thoughtHistoryLength\": 1\n}"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
The Claude Code transcript files contain a wealth of contextual data that is currently unused by the memory worker. By extracting:
|
||||
|
||||
1. User intent (the "what" and "why")
|
||||
2. Assistant reasoning (the "how" and "because")
|
||||
3. Tool results (the "outcome")
|
||||
4. Conversation context (the "flow")
|
||||
5. Session metadata (the "when" and "where")
|
||||
|
||||
We can generate significantly richer, more useful observations that better capture the intent, decisions, and outcomes of each coding session.
|
||||
|
||||
**The data is already there - we just need to read it.**
|
||||
@@ -0,0 +1,234 @@
|
||||
# Claude Code Transcript Data Discovery
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document details findings from implementing a validated transcript parser for Claude Code JSONL transcripts. The parser enables extraction of rich contextual data that can optimize prompt generation and track token usage for ROI metrics.
|
||||
|
||||
## Transcript Structure
|
||||
|
||||
### File Location
|
||||
```
|
||||
~/.claude/projects/<encoded-project-path>/<session-id>.jsonl
|
||||
```
|
||||
|
||||
Example:
|
||||
```
|
||||
~/.claude/projects/-Users-alexnewman-Scripts-claude-mem/2933cff9-f0a7-4f0b-8296-0a030e7658a6.jsonl
|
||||
```
|
||||
|
||||
### Entry Types
|
||||
|
||||
Discovered 5 transcript entry types:
|
||||
|
||||
1. **`file-history-snapshot`** (NEW - not in Python model)
|
||||
- Purpose: Track file state snapshots
|
||||
- Frequency: ~10 entries per session
|
||||
|
||||
2. **`user`** - User messages and tool results
|
||||
- Contains actual user text messages OR tool result data
|
||||
- Can have string content or array of ContentItems
|
||||
|
||||
3. **`assistant`** - Assistant responses and tool uses
|
||||
- Contains text responses, tool uses, and thinking blocks
|
||||
- **Critical**: Contains usage data with token counts
|
||||
|
||||
4. **`summary`** (not yet observed in test data)
|
||||
- Session summaries
|
||||
|
||||
5. **`system`** (not yet observed in test data)
|
||||
- System messages/warnings
|
||||
|
||||
6. **`queue-operation`** (not yet observed in test data)
|
||||
- Queue tracking for message flow
|
||||
|
||||
## Key Findings
|
||||
|
||||
### 1. Message Extraction Complexity
|
||||
|
||||
**Problem**: Naively getting the "last" entry doesn't work because:
|
||||
- Last user entry might be a tool result, not a text message
|
||||
- Last assistant entry might only contain tool uses, no text
|
||||
|
||||
**Solution**: Iterate backward through entries to find the last entry with actual text content.
|
||||
|
||||
### 2. Tool Use Tracking
|
||||
|
||||
**Discovery**: Tool uses are in **assistant** messages, not user messages.
|
||||
|
||||
**Data Available**:
|
||||
```typescript
|
||||
{
|
||||
name: string; // Tool name (e.g., "Bash", "Read", "TodoWrite")
|
||||
timestamp: string; // When the tool was used
|
||||
input: any; // Full tool input parameters
|
||||
}
|
||||
```
|
||||
|
||||
**Test Session Results** (168 entries):
|
||||
- 42 tool uses across 7 different tool types
|
||||
- Most used: Bash (24x), TodoWrite (5x), Edit (4x)
|
||||
|
||||
### 3. Token Usage Data (ROI Foundation)
|
||||
|
||||
**Critical Discovery**: Every assistant message contains complete token usage data:
|
||||
|
||||
```typescript
|
||||
interface UsageInfo {
|
||||
input_tokens?: number; // Total input tokens (includes context)
|
||||
cache_creation_input_tokens?: number; // Tokens used to create cache
|
||||
cache_read_input_tokens?: number; // Cached tokens read (discounted cost)
|
||||
output_tokens?: number; // Model output tokens
|
||||
}
|
||||
```
|
||||
|
||||
**Test Session Token Analysis**:
|
||||
```
|
||||
Input tokens: 858
|
||||
Output tokens: 44,165
|
||||
Cache creation tokens: 469,650
|
||||
Cache read tokens: 5,294,101 ← 5.29M tokens saved by caching!
|
||||
Total tokens: 45,023
|
||||
```
|
||||
|
||||
**ROI Implication**: This validates our ROI implementation plan. We can track:
|
||||
- Discovery cost = sum of all input + output tokens across session
|
||||
- Context savings = cache_read_input_tokens (tokens NOT paid for in full)
|
||||
- ROI = Discovery cost / Context savings
|
||||
|
||||
### 4. Parse Reliability
|
||||
|
||||
**Result**: 0.00% parse failure rate on production transcript with 168 entries.
|
||||
|
||||
**Conclusion**: The JSONL format is stable and well-formed. No need for extensive error handling.
|
||||
|
||||
## Implementation Files
|
||||
|
||||
### Created Files
|
||||
|
||||
1. **`src/types/transcript.ts`** - TypeScript types matching Python Pydantic model
|
||||
- All entry types, content types, usage info
|
||||
- Drop-in compatible with Python model structure
|
||||
|
||||
2. **`src/utils/transcript-parser.ts`** - Robust transcript parsing class
|
||||
- Handles all entry types
|
||||
- Smart message extraction (finds last text message, not just last entry)
|
||||
- Tool use history extraction
|
||||
- Token usage aggregation
|
||||
- Parse statistics and error tracking
|
||||
|
||||
3. **`scripts/test-transcript-parser.ts`** - Validation script
|
||||
- Tests all extraction methods
|
||||
- Reports parse statistics
|
||||
- Shows token usage breakdown
|
||||
- Lists tool use history
|
||||
|
||||
### Usage Example
|
||||
|
||||
```typescript
|
||||
import { TranscriptParser } from '../src/utils/transcript-parser.js';
|
||||
|
||||
const parser = new TranscriptParser('/path/to/transcript.jsonl');
|
||||
|
||||
// Extract messages
|
||||
const lastUserMsg = parser.getLastUserMessage();
|
||||
const lastAssistantMsg = parser.getLastAssistantMessage();
|
||||
|
||||
// Get tool history
|
||||
const tools = parser.getToolUseHistory();
|
||||
// => [{name: 'Bash', timestamp: '...', input: {...}}, ...]
|
||||
|
||||
// Get token usage
|
||||
const tokens = parser.getTotalTokenUsage();
|
||||
// => {inputTokens: 858, outputTokens: 44165, cacheReadTokens: 5294101, ...}
|
||||
|
||||
// Parse statistics
|
||||
const stats = parser.getParseStats();
|
||||
// => {totalLines: 168, parsedEntries: 168, failedLines: 0, ...}
|
||||
```
|
||||
|
||||
## Next Steps for PR Review
|
||||
|
||||
### Addressing "Drops Unknown Lines" Concern
|
||||
|
||||
**Original Issue**: Summary hook silently skipped malformed lines without visibility.
|
||||
|
||||
**Root Cause**: We didn't understand the full transcript model. The "skip malformed lines" was a band-aid.
|
||||
|
||||
**Solution**: Replace ad-hoc parsing in `summary-hook.ts` with validated `TranscriptParser` class:
|
||||
|
||||
**Before** (summary-hook.ts:38-117):
|
||||
```typescript
|
||||
// Manually parsing with try/catch, no type safety
|
||||
for (let i = lines.length - 1; i >= 0; i--) {
|
||||
try {
|
||||
const line = JSON.parse(lines[i]);
|
||||
if (line.type === 'user' && line.message?.content) {
|
||||
// ... extraction logic
|
||||
}
|
||||
} catch (parseError) {
|
||||
// Skip malformed lines ← BLACK HOLE
|
||||
continue;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**After** (using TranscriptParser):
|
||||
```typescript
|
||||
import { TranscriptParser } from '../utils/transcript-parser.js';
|
||||
|
||||
const parser = new TranscriptParser(transcriptPath);
|
||||
const lastUserMessage = parser.getLastUserMessage();
|
||||
const lastAssistantMessage = parser.getLastAssistantMessage();
|
||||
|
||||
// Parse errors are tracked in parser.getParseErrors()
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
1. ✅ Type-safe extraction based on validated model
|
||||
2. ✅ No silent failures - parse errors are tracked
|
||||
3. ✅ Smart extraction (finds last TEXT message, not last entry)
|
||||
4. ✅ Reusable across all hooks and scripts
|
||||
5. ✅ Enables token usage tracking (ROI metrics)
|
||||
6. ✅ Enables tool use tracking (prompt optimization)
|
||||
|
||||
## Prompt Optimization Opportunities
|
||||
|
||||
With rich transcript data available, we can enhance prompts with:
|
||||
|
||||
### 1. Tool Use Patterns
|
||||
- "In this session you've used: Bash (24x), TodoWrite (5x), Edit (4x)"
|
||||
- Helps Claude understand what kind of work is being done
|
||||
|
||||
### 2. Token Economics Awareness
|
||||
- "Cache read tokens: 5.29M (context savings)"
|
||||
- Reinforces value of memory system
|
||||
|
||||
### 3. Session Flow Understanding
|
||||
- Number of user/assistant exchanges
|
||||
- Tools used per exchange
|
||||
- Session complexity metrics
|
||||
|
||||
### 4. File History Snapshots
|
||||
- Track which files were modified during session
|
||||
- Provide file change context to summaries
|
||||
|
||||
## Testing
|
||||
|
||||
Run the validation script:
|
||||
```bash
|
||||
# Find your current session transcript
|
||||
ls -lt ~/.claude/projects/-Users-alexnewman-Scripts-claude-mem/*.jsonl | head -1
|
||||
|
||||
# Test the parser
|
||||
npx tsx scripts/test-transcript-parser.ts <path-to-transcript.jsonl>
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
The transcript parser implementation:
|
||||
1. ✅ Addresses PR review concern about dropped lines
|
||||
2. ✅ Validates the ROI metrics implementation plan
|
||||
3. ✅ Enables prompt optimization with rich context
|
||||
4. ✅ Provides foundation for future enhancements
|
||||
|
||||
**Recommendation**: Replace ad-hoc transcript parsing in hooks with `TranscriptParser` class for improved reliability and feature richness.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Transcript Dump
|
||||
|
||||
Total entries: 384
|
||||
|
||||
---
|
||||
|
||||
## Entry 151: USER
|
||||
**Timestamp:** 2025-11-13T17:09:47.231Z
|
||||
|
||||
**Content:**
|
||||
```
|
||||
I have a question regarding the context timeline that we made. and it's a bit of a thinker...
|
||||
|
||||
when i first started out working on claude-mem my idea was using a fast index of memories with links to access them on demand. I thought that claude-mem would be searching for all the things it needed to get the job done, but what I found is that the description of the work done + the files touched actually improved performance tremendously, and without actively searching for things constantly.
|
||||
|
||||
I just had a thought that maybe claude isn't searching, because when it reads what it did, it already "knows" how it did that, for the most part. Would you say I'm on the right track? I want you to ultrathink about WHY claude-mem's session start context works so well, from your OWN perspective
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
||||
_Remaining 364 entries omitted for brevity_
|
||||
@@ -0,0 +1,106 @@
|
||||
# Transcript Context Analysis
|
||||
|
||||
**File:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d.jsonl
|
||||
**Parsed:** 11/13/2025, 12:34:19 PM
|
||||
|
||||
## Statistics
|
||||
|
||||
- Total entries: 281
|
||||
- Successfully parsed: 281
|
||||
- Failed lines: 0
|
||||
- Conversation turns: 43
|
||||
|
||||
## Token Usage
|
||||
|
||||
- Input tokens: 6,805
|
||||
- Output tokens: 38,209
|
||||
- Cache creation: 224,078
|
||||
- Cache read: 3,320,726
|
||||
- Total: 45,014
|
||||
|
||||
---
|
||||
|
||||
# Conversation Turns
|
||||
|
||||
## Turn 1
|
||||
|
||||
---
|
||||
|
||||
## Turn 2
|
||||
|
||||
---
|
||||
|
||||
## Turn 3
|
||||
|
||||
---
|
||||
|
||||
## Turn 4
|
||||
|
||||
---
|
||||
|
||||
## Turn 5
|
||||
|
||||
---
|
||||
|
||||
## Turn 6
|
||||
|
||||
---
|
||||
|
||||
## Turn 7
|
||||
|
||||
---
|
||||
|
||||
## Turn 8
|
||||
|
||||
---
|
||||
|
||||
## Turn 9
|
||||
|
||||
---
|
||||
|
||||
## Turn 10
|
||||
|
||||
---
|
||||
|
||||
## Turn 11
|
||||
|
||||
---
|
||||
|
||||
## Turn 12
|
||||
|
||||
---
|
||||
|
||||
## Turn 13
|
||||
|
||||
---
|
||||
|
||||
## Turn 14
|
||||
|
||||
---
|
||||
|
||||
## Turn 15
|
||||
|
||||
---
|
||||
|
||||
## Turn 16
|
||||
|
||||
---
|
||||
|
||||
## Turn 17
|
||||
|
||||
---
|
||||
|
||||
## Turn 18
|
||||
|
||||
---
|
||||
|
||||
## Turn 19
|
||||
|
||||
---
|
||||
|
||||
## Turn 20
|
||||
|
||||
---
|
||||
|
||||
|
||||
_... 23 more turns omitted for brevity_
|
||||
@@ -0,0 +1,67 @@
|
||||
# VS Code API – Language Model (`lm`) Namespace
|
||||
|
||||
Excerpt captured from `https://code.visualstudio.com/api/references/vscode-api#lm` using markitdown MCP on 2025-11-12.
|
||||
|
||||
## Overview
|
||||
|
||||
The `vscode.lm` namespace exposes APIs for interacting with language models inside Visual Studio Code. It allows extensions to register tools, select chat models, invoke tools, and surface MCP servers so that agent mode can compose complex responses.
|
||||
|
||||
### Available Tools
|
||||
|
||||
- `vscode.lm.tools`: Readonly array of [`LanguageModelToolInformation`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelToolInformation).
|
||||
- Lists all tools registered via `vscode.lm.registerTool`.
|
||||
- Tools can be invoked programmatically with `vscode.lm.invokeTool` when their inputs satisfy the declared schema.
|
||||
|
||||
### Events
|
||||
|
||||
- `vscode.lm.onDidChangeChatModels`: Fires when the set of available chat models changes. Extensions should re-query models after this event.
|
||||
|
||||
### Functions
|
||||
|
||||
#### `vscode.lm.invokeTool(name, options, token?)`
|
||||
|
||||
Invokes a tool by name with a given input payload.
|
||||
|
||||
- Validates input against the schema declared by the tool.
|
||||
- When called from a chat participant, pass the `toolInvocationToken` so the chat UI associates results with the correct conversation.
|
||||
- Returns a [`LanguageModelToolResult`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelToolResult) composed of text and optional prompt-tsx parts.
|
||||
- Tool results can be preserved across turns by storing them in `ChatResult.metadata` and retrieving them later from `ChatResponseTurn.result`.
|
||||
|
||||
#### `vscode.lm.registerLanguageModelChatProvider(vendor, provider)`
|
||||
|
||||
Registers a [`LanguageModelChatProvider`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelChatProvider).
|
||||
|
||||
- Requires a matching `languageModelChatProviders` contribution in `package.json`.
|
||||
- `vendor` must be globally unique (for example `copilot` or `openai`).
|
||||
- Returns a `Disposable` to unregister the provider.
|
||||
|
||||
#### `vscode.lm.registerMcpServerDefinitionProvider(id, provider)`
|
||||
|
||||
Publishes Model Context Protocol servers for the editor.
|
||||
|
||||
- Requires a `contributes.mcpServerDefinitionProviders` entry in `package.json`.
|
||||
- Enables dynamic discovery of MCP servers and tools when users submit chat messages.
|
||||
- Returns a `Disposable` that unregisters the provider.
|
||||
|
||||
#### `vscode.lm.registerTool(name, tool)`
|
||||
|
||||
Registers a [`LanguageModelTool`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelTool) implementation with the runtime.
|
||||
|
||||
- Tool must also appear in `package.json -> contributes.languageModelTools`.
|
||||
- Registered tools appear in `vscode.lm.tools` and can be used by any extension.
|
||||
|
||||
#### `vscode.lm.selectChatModels(selector?)`
|
||||
|
||||
Returns an array of [`LanguageModelChat`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelChat) instances matching a selector.
|
||||
|
||||
- Selector can be broad (by vendor or family) or narrow (by exact model ID).
|
||||
- Handle scenarios where zero models are available.
|
||||
- Persisted model references should be refreshed when `onDidChangeChatModels` fires.
|
||||
|
||||
## Usage Notes
|
||||
|
||||
- Extensions should gracefully handle missing models or tools.
|
||||
- Tool invocation responses can include multiple parts; integrate them using prompt-tsx or by constructing `LanguageModelToolResultPart` objects.
|
||||
- When providing MCP servers or tools, ensure proper contribution points exist in `package.json`.
|
||||
|
||||
_Fetched on 2025-11-12 via markitdown MCP._
|
||||
@@ -0,0 +1,49 @@
|
||||
# VS Code Copilot Extension Integration Plan
|
||||
|
||||
## 1. Groundwork
|
||||
- Audit existing claude-mem hook scripts (`context-hook`, `user-message-hook`, `new-hook`, `save-hook`, `summary-hook`, `cleanup-hook`) and their worker-service payloads.
|
||||
- Document REST endpoints, request bodies, and SessionStore schema fields used today so the extension mirrors them exactly.
|
||||
- Confirm worker service availability workflow (`ensureWorkerRunning`, port resolution) and decide how extension error reporting will surface issues to Copilot chat users.
|
||||
|
||||
## 2. Project Scaffold
|
||||
- Clone the VS Code `chat-sample` starter, convert to a TypeScript-only extension, and align lint/tsconfig with repo standards.
|
||||
- Add build pipeline (esbuild or webpack) plus npm scripts that match the existing `scripts/build-hooks.js` release flow.
|
||||
- Wire extension activation events for chat participation and ensure packaging metadata (publisher, categories) is in place.
|
||||
|
||||
## 3. Shared Worker Client
|
||||
- Extract reusable worker-service client utilities from `plugin/scripts/*.js` (port discovery, session init, observation uploads).
|
||||
- Publish TypeScript definitions by re-exporting from `src/services/worker-types.ts` to keep contracts synchronized.
|
||||
- Centralize HTTP calls (timeouts, retries, logging) so every tool implementation uses the same helper layer.
|
||||
|
||||
## 4. Language Model Tool Contracts
|
||||
- Add `contributes.languageModelTools` entries in `package.json` for lifecycle parity:
|
||||
- `mem_session_init`, `mem_user_prompt_log`, `mem_observation_record`, `mem_summary_finalize`, `mem_session_cleanup`.
|
||||
- Provide detailed JSON schemas mirroring hook input structures (session IDs, cwd, prompt text, tool payload metadata).
|
||||
- Supply descriptive `modelDescription`, `userDescription`, icons, tags, and enable `canBeReferencedInPrompt` where appropriate.
|
||||
|
||||
## 5. Tool Implementations
|
||||
- Register each tool via `vscode.lm.registerTool` inside `activate`.
|
||||
- Implement `prepareInvocation` to show user confirmations (especially for cleanup/stop actions) and tailor messages to match existing CLI prompts.
|
||||
- In `invoke`, call the shared worker client, translate successes into `LanguageModelToolResult` text parts, and craft error messages that guide the LLM toward recovery (retry, alternate parameters).
|
||||
- Ensure telemetry/logging records tool usage for debugging without leaking sensitive data.
|
||||
|
||||
## 6. Chat Orchestration
|
||||
- Implement a chat participant based on the sample that maps Copilot threads to claude-mem session IDs stored in turn metadata.
|
||||
- On conversation start, auto-run `mem_session_init`; before each user prompt, dispatch `mem_user_prompt_log`; when Copilot signals stop, run `mem_summary_finalize` (with fallbacks if the worker is unavailable).
|
||||
- Capture tool events emitted by Copilot (file edits, terminal runs) and forward them through `mem_observation_record` with matching payload structure.
|
||||
- Handle conversation disposal or model changes by calling `mem_session_cleanup` to mirror `SessionEnd` hooks.
|
||||
|
||||
## 7. Settings and UX
|
||||
- Read `.claude-mem/settings.json` overrides (worker port, observation depth) and surface VS Code settings for Copilot-specific toggles (auto-sync enabled, max observations per prompt).
|
||||
- Add status bar indicator/commands for worker health, quick restart instructions, and opening the viewer UI (`http://localhost:37777`).
|
||||
- Provide inline notifications when the worker is unreachable, including guidance to restart via PM2.
|
||||
|
||||
## 8. Testing and QA
|
||||
- Draft manual validation checklist: initial session, prompt logging, observation capture, summary completion, worker-down handling.
|
||||
- Add integration tests using `@vscode/test-electron` to simulate chat turns and assert database side effects in a temporary claude-mem data directory.
|
||||
- Build mocks for worker endpoints to enable unit tests of tool invocation logic without hitting the real service.
|
||||
|
||||
## 9. Release Readiness
|
||||
- Document installation and usage in `README.md`, including architecture diagrams showing Copilot → tool → worker flow.
|
||||
- Update CHANGELOG and marketing copy to announce Copilot support and list prerequisites (worker running, settings file placement).
|
||||
- Prepare Marketplace assets (icon, gallery text) and extend existing publish scripts to package and ship the new extension.
|
||||
@@ -0,0 +1,8 @@
|
||||
# VS Code Extension Chat Sample Repository (GitHub Snapshot)
|
||||
|
||||
The markitdown MCP fetch for `https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample` returned only the public navigation scaffolding for GitHub. No repository-specific content or README data was captured because the site requires client-side execution that the fetcher cannot perform.
|
||||
|
||||
You can browse the repository directly for full details:
|
||||
- https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample
|
||||
|
||||
_Fetched on 2025-11-12 via markitdown MCP. Content retrieval was limited to GitHub's static navigation shell._
|
||||
@@ -0,0 +1,267 @@
|
||||
# Language Model Tool API
|
||||
|
||||
Language model tools enable you to extend the functionality of a large language model (LLM) in chat with domain-specific capabilities. To process a user's chat prompt, [agent mode](/docs/copilot/chat/chat-agent-mode) in VS Code can automatically invoke these tools to perform specialized tasks as part of the conversation.
|
||||
|
||||
By contributing a language model tool in your VS Code extension, you can extend the agentic coding workflow while also providing deep integration with the editor. Extension tools are one of three types of tools available in VS Code, alongside [built-in tools and MCP tools](/docs/copilot/chat/chat-tools.md#types-of-tools).
|
||||
|
||||
In this extension guide, you learn how to create a language model tool by using the Language Model Tools API and how to implement tool calling in a chat extension.
|
||||
|
||||
You can also extend the chat experience with specialized tools by contributing an [MCP server](/api/extension-guides/ai/mcp). See the [AI Extensibility Overview](/api/extension-guides/ai/ai-extensibility-overview) for details on the different options and how to decide which approach to use.
|
||||
|
||||
> **Tip**
|
||||
> For information about using tools as an end user, see [Use tools in chat](/docs/copilot/chat/chat-tools.md).
|
||||
|
||||
## What is tool calling in an LLM?
|
||||
|
||||
A language model tool is a function that can be invoked as part of a language model request. For example, you might have a function that retrieves information from a database, performs some calculation, or calls an online API. When you contribute a tool in a VS Code extension, agent mode can then invoke the tool based on the context of the conversation.
|
||||
|
||||
The LLM never actually executes the tool itself, instead the LLM generates the parameters that are used to call your tool. It's important to clearly describe the tool's purpose, functionality, and input parameters so that the tool can be invoked in the right context.
|
||||
|
||||
The following diagram shows the tool-calling flow in agent mode in VS Code. See [Tool-calling flow](#tool-calling-flow) for details about the specific steps involved.
|
||||
|
||||

|
||||
|
||||
Read more about [function calling](https://platform.openai.com/docs/guides/function-calling) in the OpenAI documentation.
|
||||
|
||||
## Why implement a language model tool in your extension?
|
||||
|
||||
Implementing a language model tool in your extension has several benefits:
|
||||
|
||||
- **Extend agent mode** with specialized, domain-specific tools that are automatically invoked as part of responding to a user prompt. For example, enable database scaffolding and querying to dynamically provide the LLM with relevant context.
|
||||
- **Deeply integrate with VS Code** by using the broad set of extension APIs. For example, use the [debug APIs](/api/extension-guides/debugger-extension) to get the current debugging context and use it as part of the tool's functionality.
|
||||
- **Distribute and deploy** tools via the Visual Studio Marketplace, providing a reliable and seamless experience for users. Users don't need a separate installation and update process for your tool.
|
||||
|
||||
You might consider implementing a language model tool with an [MCP server](/api/extension-guides/ai/mcp) in the following scenarios:
|
||||
|
||||
- You already have an MCP server implementation and also want to use it in VS Code.
|
||||
- You want to reuse the same tool across different development environments and platforms.
|
||||
- Your tool is hosted remotely as a service.
|
||||
- You don't need access to VS Code APIs.
|
||||
|
||||
Learn more about the [differences between tool types](/docs/copilot/chat/chat-tools.md#types-of-tools).
|
||||
|
||||
## Create a language model tool
|
||||
|
||||
Implementing a language model tool consists of two main parts:
|
||||
|
||||
1. Define the tool's configuration in the `package.json` file of your extension.
|
||||
2. Implement the tool in your extension code by using the [Language Model API reference](/api/references/vscode-api#lm)
|
||||
|
||||
You can get started with a [basic example project](https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample).
|
||||
|
||||
### 1. Static configuration in `package.json`
|
||||
|
||||
The first step to define a language model tool in your extension is to define it in the `package.json` file of your extension. This configuration includes the tool name, description, input schema, and other metadata:
|
||||
|
||||
1. Add an entry for your tool in the `contributes.languageModelTools` section of your extension's `package.json` file.
|
||||
2. Give the tool a unique name:
|
||||
|
||||
| Property | Description |
|
||||
| --- | --- |
|
||||
| `name` | The unique name of the tool, used to reference the tool in the extension implementation code. Format the name in the format `{verb}_{noun}`. See [naming guidelines](#guidelines-and-conventions). |
|
||||
| `displayName` | The user-friendly name of the tool, used for displaying in the UI. |
|
||||
|
||||
3. If the tool can be used in [agent mode](/docs/copilot/chat/chat-agent-mode) or referenced in a chat prompt with `#`, add the following properties:
|
||||
|
||||
Users can enable or disable the tool in the Chat view, similar to how this is done for [Model Context Protocol (MCP) tools](/docs/copilot/chat/chat-tools.md#mcp-tools).
|
||||
|
||||
| Property | Description |
|
||||
| --- | --- |
|
||||
| `canBeReferencedInPrompt` | Set to `true` if the tool can be used in [agent mode](/docs/copilot/chat/chat-agent-mode) or referenced in chat. |
|
||||
| `toolReferenceName` | The name for users to reference the tool in a chat prompt via `#`. |
|
||||
| `icon` | The icon to display for the tool in the UI. |
|
||||
| `userDescription` | User-friendly description of the tool, used for displaying in the UI. |
|
||||
|
||||
4. Add a detailed description in `modelDescription`. This information is used by the LLM to determine in which context your tool should be used.
|
||||
|
||||
- What exactly does the tool do?
|
||||
- What kind of information does it return?
|
||||
- When should and shouldn't it be used?
|
||||
- Describe important limitations or constraints of the tool.
|
||||
|
||||
5. If the tool takes input parameters, add an `inputSchema` property that describes the tool's input parameters.
|
||||
|
||||
This JSON schema describes an object with the properties that the tool takes as input, and whether they are required. File paths should be absolute paths.
|
||||
|
||||
Describe what each parameter does and how it relates to the tool's functionality.
|
||||
|
||||
6. Add a `when` clause to control when the tool is available.
|
||||
|
||||
The `languageModelTools` contribution point lets you restrict when a tool is available for agent mode or can be referenced in a prompt by using a [when clause](/api/references/when-clause-contexts). For example, a tool that gets the debug call stack information should only be available when the user is debugging.
|
||||
|
||||
```json
|
||||
"contributes": {
|
||||
"languageModelTools": [
|
||||
{
|
||||
"name": "chat-tools-sample_tabCount",
|
||||
...
|
||||
"when": "debugState == 'running'"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Example tool definition**
|
||||
|
||||
The following example shows how to define a tool that counts the number of active tabs in a tab group.
|
||||
|
||||
```json
|
||||
"contributes": {
|
||||
"languageModelTools": [
|
||||
{
|
||||
"name": "chat-tools-sample_tabCount",
|
||||
"tags": [
|
||||
"editors",
|
||||
"chat-tools-sample"
|
||||
],
|
||||
"toolReferenceName": "tabCount",
|
||||
"displayName": "Tab Count",
|
||||
"modelDescription": "The number of active tabs in a tab group in VS Code.",
|
||||
"userDescription": "Count the number of active tabs in a tab group.",
|
||||
"canBeReferencedInPrompt": true,
|
||||
"icon": "$(files)",
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"tabGroup": {
|
||||
"type": "number",
|
||||
"description": "The index of the tab group to check. This is optional- if not specified, the active tab group will be checked.",
|
||||
"default": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Tool implementation
|
||||
|
||||
Implement the language model tool by using the [Language Model API](/api/references/vscode-api#lm). This consists of the following steps:
|
||||
|
||||
1. On activation of the extension, register the tool with [`vscode.lm.registerTool`](/api/references/vscode-api#lm.registerTool).
|
||||
|
||||
Provide the name of the tool as you specified it in the `name` property in `package.json`.
|
||||
|
||||
If you want the tool to be private to your extension, skip the tool registration step.
|
||||
|
||||
```ts
|
||||
export function registerChatTools(context: vscode.ExtensionContext) {
|
||||
context.subscriptions.push(
|
||||
vscode.lm.registerTool('chat-tools-sample_tabCount', new TabCountTool())
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
2. Create a class that implements the [`vscode.LanguageModelTool<>`](/api/references/vscode-api#LanguageModelTool%3CT%3E) interface.
|
||||
|
||||
3. Add tool confirmation messages in the `prepareInvocation` method.
|
||||
|
||||
A generic confirmation dialog will always be shown for tools from extensions, but the tool can customize the confirmation message. Give enough context to the user to understand what the tool is doing. The message can be a `MarkdownString` containing a code block.
|
||||
|
||||
The following example shows how to provide a confirmation message for the tab count tool.
|
||||
|
||||
```ts
|
||||
async prepareInvocation(
|
||||
options: vscode.LanguageModelToolInvocationPrepareOptions<ITabCountParameters>,
|
||||
_token: vscode.CancellationToken
|
||||
) {
|
||||
const confirmationMessages = {
|
||||
title: 'Count the number of open tabs',
|
||||
message: new vscode.MarkdownString(
|
||||
`Count the number of open tabs?` +
|
||||
(options.input.tabGroup !== undefined
|
||||
? ` in tab group ${options.input.tabGroup}`
|
||||
: '')
|
||||
),
|
||||
};
|
||||
|
||||
return {
|
||||
invocationMessage: 'Counting the number of tabs',
|
||||
confirmationMessages,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
If `prepareInvocation` returned `undefined`, the generic confirmation message will be shown. Note that the user can also select to "Always Allow" a certain tool.
|
||||
|
||||
4. Define an interface that describes the tool input parameters.
|
||||
|
||||
The interface is used in the `invoke` method of the `vscode.LanguageModelTool` class. The input parameters are validated against the JSON schema you defined in the `inputSchema` in `package.json`.
|
||||
|
||||
The following example shows the interface for the tab count tool.
|
||||
|
||||
```ts
|
||||
export interface ITabCountParameters {
|
||||
tabGroup?: number;
|
||||
}
|
||||
```
|
||||
|
||||
5. Implement the `invoke` method. This method is called when the language model tool is invoked while processing a chat prompt.
|
||||
|
||||
The `invoke` method receives the tool input parameters in the `options` parameter. The parameters are validated against the JSON schema defined in `inputSchema` in `package.json`.
|
||||
|
||||
When an error occurs, throw an error with a message that makes sense to the LLM. Optionally, provide instructions on what the LLM should do next, such as retrying with different parameters, or performing a different action.
|
||||
|
||||
The following example shows the implementation of the tab count tool. The result of the tool is an instance of type `vscode.LanguageModelToolResult`.
|
||||
|
||||
```ts
|
||||
async invoke(
|
||||
options: vscode.LanguageModelToolInvocationOptions<ITabCountParameters>,
|
||||
_token: vscode.CancellationToken
|
||||
) {
|
||||
const params = options.input;
|
||||
if (typeof params.tabGroup === 'number') {
|
||||
const group = vscode.window.tabGroups.all[Math.max(params.tabGroup - 1, 0)];
|
||||
const nth =
|
||||
params.tabGroup === 1
|
||||
? '1st'
|
||||
: params.tabGroup === 2
|
||||
? '2nd'
|
||||
: params.tabGroup === 3
|
||||
? '3rd'
|
||||
: `${params.tabGroup}th`;
|
||||
return new vscode.LanguageModelToolResult([new vscode.LanguageModelTextPart(`There are ${group.tabs.length} tabs open in the ${nth} tab group.`)]);
|
||||
} else {
|
||||
const group = vscode.window.tabGroups.activeTabGroup;
|
||||
return new vscode.LanguageModelToolResult([new vscode.LanguageModelTextPart(`There are ${group.tabs.length} tabs open.`)]);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
View the full source code for implementing a [language model tool](https://github.com/microsoft/vscode-extension-samples/blob/main/chat-sample/src/tools.ts) in the VS Code Extension Samples repository.
|
||||
|
||||
## Tool-calling flow
|
||||
|
||||
When a user sends a chat prompt, the following steps occur:
|
||||
|
||||
1. Copilot determines the list of available tools based on the user's configuration.
|
||||
The list of tools consists of built-in tools, tools registered by extensions, and tools from [MCP servers](/docs/copilot/chat/mcp-servers). You can contribute to agent mode via extensions or MCP servers (shown in green in the diagram).
|
||||
2. Copilot sends the request to the LLM and provides it with the prompt, chat context, and the list of tool definitions to consider.
|
||||
The LLM generates a response, which might include one or more requests to invoke a tool.
|
||||
3. If needed, Copilot invokes the suggested tool(s) with the parameter values provided by the LLM.
|
||||
A tool response might result in more requests for tool invocations.
|
||||
4. If there are errors or follow-up tool requests, Copilot iterates over the tool-calling flow until all tool requests are resolved.
|
||||
5. Copilot returns the final response to the user, which might include responses from multiple tools.
|
||||
|
||||
## Guidelines and conventions
|
||||
|
||||
- **Naming**: write clear and descriptive names for tools and parameters.
|
||||
- **Tool name**: should be unique, and clearly describe their intent. Structure the tool name in the format `{verb}_{noun}`. For example, `get_weather`, `get_azure_deployment`, or `get_terminal_output`.
|
||||
- **Parameter name**: should describe the parameter's purpose. Structure the parameter name in the format `{noun}`. For example, `destination_location`, `ticker`, or `file_name`.
|
||||
- **Descriptions**: write detailed descriptions for tools and parameters.
|
||||
- Describe what the tool does and when it should and shouldn't be used. For example, "This tool retrieves the weather for a given location."
|
||||
- Describe what each parameter does and how it relates to the tool's functionality. For example, "The `destination_location` parameter specifies the location for which to retrieve the weather. It should be a valid location name or coordinates."
|
||||
- Describe important limitations or constraints of the tool. For example, "This tool only retrieves weather data for locations in the United States. It might not work for other regions."
|
||||
- **User confirmation**: provide a confirmation message for the tool invocation. A generic confirmation dialog will always be shown for tools from extensions, but the tool can customize the confirmation message. Give enough context to the user to understand what the tool is doing.
|
||||
- **Error handling**: when an error occurs, throw an error with a message that makes sense to the LLM. Optionally, provide instructions on what the LLM should do next, such as retrying with different parameters, or performing a different action.
|
||||
|
||||
Get more best practices for creating tools in the [OpenAI documentation](https://platform.openai.com/docs/guides/function-calling?api-mode=chat#best-practices-for-defining-functions) and [Anthropic documentation](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview).
|
||||
|
||||
## Related content
|
||||
|
||||
- [Language Model API reference](/api/references/vscode-api#lm)
|
||||
- [Register an MCP server in a VS Code extension](/api/extension-guides/ai/mcp)
|
||||
- [Use MCP tools in agent mode](/docs/copilot/chat/mcp-servers)
|
||||
|
||||
_Fetched on 2025-11-12 via markitdown MCP._
|
||||
@@ -0,0 +1,13 @@
|
||||
VSCode Language Model Tool API
|
||||
|
||||
Local snapshots fetched via the markitdown MCP on 2025-11-12:
|
||||
|
||||
- `docs/context/vscode-language-model-tool-api.md`
|
||||
- `docs/context/vscode-extension-chat-sample.md`
|
||||
- `docs/context/vscode-api-lm.md`
|
||||
|
||||
Original sources for reference:
|
||||
|
||||
- http://code.visualstudio.com/api/extension-guides/ai/tools
|
||||
- https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample
|
||||
- https://code.visualstudio.com/api/references/vscode-api#lm
|
||||
@@ -0,0 +1,88 @@
|
||||
# Claude-Mem Public Documentation
|
||||
|
||||
## What This Folder Is
|
||||
|
||||
This `docs/public/` folder contains the **Mintlify documentation site** - the official user-facing documentation for claude-mem. It's a structured documentation platform with a specific file format and organization.
|
||||
|
||||
## Folder Structure
|
||||
|
||||
```
|
||||
docs/
|
||||
├── public/ ← You are here (Mintlify MDX files)
|
||||
│ ├── *.mdx - User-facing documentation pages
|
||||
│ ├── docs.json - Mintlify configuration and navigation
|
||||
│ ├── architecture/ - Technical architecture docs
|
||||
│ ├── usage/ - User guides and workflows
|
||||
│ └── *.webp, *.gif - Assets (logos, screenshots)
|
||||
└── context/ ← Internal documentation (DO NOT put here)
|
||||
└── *.md - Planning docs, audits, references
|
||||
```
|
||||
|
||||
## File Requirements
|
||||
|
||||
### Mintlify Documentation Files (.mdx)
|
||||
All official documentation files must be:
|
||||
- Written in `.mdx` format (Markdown with JSX support)
|
||||
- Listed in `docs.json` navigation structure
|
||||
- Follow Mintlify's schema and conventions
|
||||
|
||||
The documentation is organized into these sections:
|
||||
- **Get Started**: Introduction, installation, usage guides
|
||||
- **Best Practices**: Context engineering, progressive disclosure
|
||||
- **Configuration & Development**: Settings, dev workflow, troubleshooting
|
||||
- **Architecture**: System design, components, technical details
|
||||
|
||||
### Configuration File
|
||||
`docs.json` defines:
|
||||
- Site metadata (name, description, theme)
|
||||
- Navigation structure
|
||||
- Branding (logos, colors)
|
||||
- Footer links and social media
|
||||
|
||||
## What Does NOT Belong Here
|
||||
|
||||
**Planning documents, design docs, and reference materials go in `/docs/context/` instead:**
|
||||
|
||||
Files that belong in `/docs/context/` (NOT here):
|
||||
- Planning documents (`*-plan.md`, `*-outline.md`)
|
||||
- Implementation analysis (`*-audit.md`, `*-code-reference.md`)
|
||||
- Error tracking (`typescript-errors.md`)
|
||||
- Internal design documents
|
||||
- PR review responses
|
||||
- Reference materials (like `agent-sdk-ref.md`)
|
||||
- Work-in-progress documentation
|
||||
|
||||
## How to Add Official Documentation
|
||||
|
||||
1. Create a new `.mdx` file in the appropriate subdirectory
|
||||
2. Add the file path to `docs.json` navigation
|
||||
3. Use Mintlify's frontmatter and components
|
||||
4. Follow the existing documentation style
|
||||
5. Test locally: `npx mintlify dev`
|
||||
|
||||
## Development Workflow
|
||||
|
||||
**For contributors working on claude-mem:**
|
||||
- Read `/CLAUDE.md` in the project root for development instructions
|
||||
- Place planning/design docs in `/docs/context/`
|
||||
- Only add user-facing documentation to `/docs/public/`
|
||||
- Test documentation locally with Mintlify CLI before committing
|
||||
|
||||
## Testing Documentation
|
||||
|
||||
```bash
|
||||
# Validate docs structure
|
||||
npx mintlify validate
|
||||
|
||||
# Check for broken links
|
||||
npx mintlify broken-links
|
||||
|
||||
# Run local dev server
|
||||
npx mintlify dev
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
**Simple Rule**:
|
||||
- `/docs/public/` = Official user documentation (Mintlify .mdx files) ← YOU ARE HERE
|
||||
- `/docs/context/` = Internal docs, plans, references, audits
|
||||
@@ -1,17 +1,17 @@
|
||||
---
|
||||
title: "Plugin Hooks"
|
||||
description: "7 hook scripts that power Claude-Mem"
|
||||
description: "6 lifecycle hooks that power Claude-Mem"
|
||||
---
|
||||
|
||||
# Plugin Hooks
|
||||
|
||||
Claude-Mem integrates with Claude Code through 7 hook scripts across 5 lifecycle events that capture events and inject context.
|
||||
Claude-Mem integrates with Claude Code through 6 hook scripts across 5 lifecycle events that capture events and inject context. Additionally, a smart-install pre-hook script manages dependencies.
|
||||
|
||||
## Hook Overview
|
||||
|
||||
| Hook Name | Purpose | Timeout | Script |
|
||||
| Hook Type | Purpose | Timeout | Script |
|
||||
|---------------------|--------------------------------------|---------|-------------------------|
|
||||
| SessionStart | Smart dependency installation | 300s | smart-install.js |
|
||||
| Pre-Hook | Smart dependency installation | 300s | smart-install.js* |
|
||||
| SessionStart | Inject context from previous sessions| 300s | context-hook.js |
|
||||
| SessionStart | Display first-time setup message | 10s | user-message-hook.js |
|
||||
| UserPromptSubmit | Create/track new sessions | 120s | new-hook.js |
|
||||
@@ -19,6 +19,8 @@ Claude-Mem integrates with Claude Code through 7 hook scripts across 5 lifecycle
|
||||
| Stop | Generate session summaries | 120s | summary-hook.js |
|
||||
| SessionEnd | Mark sessions complete | 120s | cleanup-hook.js |
|
||||
|
||||
*smart-install.js is a pre-hook script (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json.
|
||||
|
||||
## Hook Configuration
|
||||
|
||||
Hooks are configured in `plugin/hooks/hooks.json`:
|
||||
@@ -72,10 +74,12 @@ Hooks are configured in `plugin/hooks/hooks.json`:
|
||||
}
|
||||
```
|
||||
|
||||
## 1. SessionStart Hook - Smart Install (`smart-install.js`)
|
||||
## 1. Pre-Hook Script - Smart Install (`smart-install.js`)
|
||||
|
||||
**Purpose**: Intelligently manage dependencies and ensure worker service is running.
|
||||
|
||||
**Note**: This is NOT a lifecycle hook - it's a pre-hook script executed via command chaining before context-hook.js runs.
|
||||
|
||||
**Behavior**:
|
||||
- Checks if dependencies need installation using version marker (`.install-version`)
|
||||
- Only runs npm install when:
|
||||
@@ -95,7 +99,7 @@ Hooks are configured in `plugin/hooks/hooks.json`:
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation**: `scripts/smart-install.js`
|
||||
**Implementation**: `scripts/smart-install.js` (standalone script, not in src/hooks/)
|
||||
|
||||
**Key Features**:
|
||||
- Version caching prevents redundant installs
|
||||
@@ -9,11 +9,12 @@ description: "System components and data flow in Claude-Mem"
|
||||
|
||||
Claude-Mem operates as a Claude Code plugin with five core components:
|
||||
|
||||
1. **Plugin Hooks** - Capture lifecycle events (7 hook files)
|
||||
2. **Worker Service** - Process observations via Claude Agent SDK + HTTP API (10 search endpoints)
|
||||
3. **Database Layer** - Store sessions and observations (SQLite + FTS5 + ChromaDB)
|
||||
4. **Search Skill** - Skill-based search with progressive disclosure (v5.4.0+)
|
||||
5. **Viewer UI** - Web-based real-time memory stream visualization
|
||||
1. **Plugin Hooks** - Capture lifecycle events (6 hook files)
|
||||
2. **Smart Install** - Cached dependency checker (pre-hook script, runs before context-hook)
|
||||
3. **Worker Service** - Process observations via Claude Agent SDK + HTTP API (10 search endpoints)
|
||||
4. **Database Layer** - Store sessions and observations (SQLite + FTS5 + ChromaDB)
|
||||
5. **mem-search Skill** - Skill-based search with progressive disclosure (v5.4.0+)
|
||||
6. **Viewer UI** - Web-based real-time memory stream visualization
|
||||
|
||||
## Technology Stack
|
||||
|
||||
@@ -46,11 +47,11 @@ Hook (stdin) → Database → Worker Service → SDK Processor → Database →
|
||||
|
||||
### Search Pipeline (v5.4.0+)
|
||||
```
|
||||
User Query → Skill Invoked → HTTP API → SessionSearch Service → FTS5 Database → Search Results → Claude
|
||||
User Query → mem-search Skill Invoked → HTTP API → SessionSearch Service → FTS5 Database → Search Results → Claude
|
||||
```
|
||||
|
||||
1. **User Query**: User asks naturally: "What bugs did we fix?"
|
||||
2. **Skill Invoked**: Claude recognizes intent and invokes search skill
|
||||
2. **Skill Invoked**: Claude recognizes intent and invokes mem-search skill
|
||||
3. **HTTP API**: Skill uses curl to call HTTP endpoint (e.g., `/api/search/observations`)
|
||||
4. **SessionSearch**: Worker service queries FTS5 virtual tables
|
||||
5. **Format**: Results formatted and returned to skill
|
||||
@@ -62,8 +63,9 @@ User Query → Skill Invoked → HTTP API → SessionSearch Service → FTS5 Dat
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 0. Smart Install Hook Fires │
|
||||
│ 0. Smart Install Pre-Hook Fires │
|
||||
│ Checks dependencies (cached), only runs on version changes │
|
||||
│ Not a lifecycle hook - runs before context-hook starts │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
@@ -104,14 +106,14 @@ User Query → Skill Invoked → HTTP API → SessionSearch Service → FTS5 Dat
|
||||
```
|
||||
claude-mem/
|
||||
├── src/
|
||||
│ ├── hooks/ # Hook implementations (7 hooks)
|
||||
│ │ ├── smart-install.ts # Dependency check (cached)
|
||||
│ ├── hooks/ # Hook implementations (6 hooks)
|
||||
│ │ ├── context-hook.ts # SessionStart
|
||||
│ │ ├── user-message-hook.ts # UserMessage (for debugging)
|
||||
│ │ ├── new-hook.ts # UserPromptSubmit
|
||||
│ │ ├── save-hook.ts # PostToolUse
|
||||
│ │ ├── summary-hook.ts # Stop
|
||||
│ │ └── cleanup-hook.ts # SessionEnd
|
||||
│ │ ├── cleanup-hook.ts # SessionEnd
|
||||
│ │ └── hook-response.ts # Hook response utilities
|
||||
│ │
|
||||
│ ├── sdk/ # Claude Agent SDK integration
|
||||
│ │ ├── prompts.ts # XML prompt builders
|
||||
@@ -143,13 +145,15 @@ claude-mem/
|
||||
│ ├── platform.ts
|
||||
│ └── port-allocator.ts
|
||||
│
|
||||
├── scripts/ # Build and utility scripts
|
||||
│ └── smart-install.js # Cached dependency checker (pre-hook)
|
||||
│
|
||||
├── plugin/ # Plugin distribution
|
||||
│ ├── .claude-plugin/
|
||||
│ │ └── plugin.json
|
||||
│ ├── hooks/
|
||||
│ │ └── hooks.json
|
||||
│ ├── scripts/ # Built executables
|
||||
│ │ ├── smart-install.js
|
||||
│ │ ├── context-hook.js
|
||||
│ │ ├── user-message-hook.js
|
||||
│ │ ├── new-hook.js
|
||||
@@ -159,11 +163,14 @@ claude-mem/
|
||||
│ │ └── worker-service.cjs # Background worker + HTTP API
|
||||
│ │
|
||||
│ ├── skills/ # Agent skills (v5.4.0+)
|
||||
│ │ ├── search/ # Search skill with progressive disclosure
|
||||
│ │ ├── mem-search/ # Search skill with progressive disclosure (v5.5.0)
|
||||
│ │ │ ├── SKILL.md # Skill frontmatter (~250 tokens)
|
||||
│ │ │ └── operations/ # Detailed operation docs
|
||||
│ │ │ ├── operations/ # 12 detailed operation docs
|
||||
│ │ │ └── principles/ # 2 principle guides
|
||||
│ │ ├── troubleshoot/ # Troubleshooting skill
|
||||
│ │ └── version-bump/ # Version management skill
|
||||
│ │ │ ├── SKILL.md
|
||||
│ │ │ └── operations/ # 6 operation docs
|
||||
│ │ └── version-bump/ # Version management skill (deprecated)
|
||||
│ │
|
||||
│ └── ui/ # Built viewer UI
|
||||
│ └── viewer.html # Self-contained bundle
|
||||
@@ -175,8 +182,7 @@ claude-mem/
|
||||
|
||||
## Component Details
|
||||
|
||||
### 1. Plugin Hooks (7 Hooks)
|
||||
- **smart-install.js** - Cached dependency checker (only runs on version changes)
|
||||
### 1. Plugin Hooks (6 Hooks)
|
||||
- **context-hook.js** - SessionStart: Starts PM2 worker, injects context
|
||||
- **user-message-hook.js** - UserMessage: Debugging hook
|
||||
- **new-hook.js** - UserPromptSubmit: Creates session, saves prompt
|
||||
@@ -184,6 +190,8 @@ claude-mem/
|
||||
- **summary-hook.js** - Stop: Generates session summary
|
||||
- **cleanup-hook.js** - SessionEnd: Marks session complete
|
||||
|
||||
**Note**: smart-install.js is a pre-hook dependency checker (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json and only runs when dependencies need updating.
|
||||
|
||||
See [Plugin Hooks](/architecture/hooks) for detailed hook documentation.
|
||||
|
||||
### 2. Worker Service
|
||||
@@ -205,7 +213,7 @@ SQLite3 with better-sqlite3 driver featuring:
|
||||
|
||||
See [Database Architecture](/architecture/database) for schema and FTS5 search.
|
||||
|
||||
### 4. Search Skill (v5.4.0+)
|
||||
### 4. mem-search Skill (v5.4.0+)
|
||||
Skill-based search with progressive disclosure providing 10 search operations:
|
||||
- Search observations, sessions, prompts (full-text FTS5)
|
||||
- Filter by type, concept, file
|
||||
@@ -217,6 +225,8 @@ Skill-based search with progressive disclosure providing 10 search operations:
|
||||
- Full instructions: ~2,500 tokens (loaded on-demand when invoked)
|
||||
- HTTP API endpoints instead of MCP tools
|
||||
|
||||
**Skill Enhancement (v5.5.0)**: Renamed from "search" to "mem-search" for better scope differentiation. Effectiveness increased from 67% to 100% with enhanced triggers and comprehensive documentation.
|
||||
|
||||
See [Search Architecture](/architecture/search-architecture) for technical details and examples.
|
||||
|
||||
### 5. Viewer UI
|
||||
@@ -1,23 +1,29 @@
|
||||
---
|
||||
title: "Search Architecture"
|
||||
description: "Skill-based search with HTTP API and progressive disclosure"
|
||||
description: "mem-search skill with HTTP API and progressive disclosure"
|
||||
---
|
||||
|
||||
# Search Architecture
|
||||
|
||||
Claude-Mem uses a skill-based search architecture that provides intelligent memory retrieval through natural language queries. This replaced the MCP-based approach in v5.4.0, saving ~2,250 tokens per session start.
|
||||
Claude-Mem uses a skill-based search architecture that provides intelligent memory retrieval through natural language queries. This replaced the MCP-based approach in v5.4.0, saving ~2,250 tokens per session start. The skill was enhanced and renamed to "mem-search" in v5.5.0 for better scope differentiation.
|
||||
|
||||
## Overview
|
||||
|
||||
**Architecture**: Skill-Based Search + HTTP API + Progressive Disclosure
|
||||
|
||||
**Key Components**:
|
||||
1. **Search Skill** (`plugin/skills/search/SKILL.md`) - Auto-invoked when users ask about past work
|
||||
1. **mem-search Skill** (`plugin/skills/mem-search/SKILL.md`) - Auto-invoked when users ask about past work
|
||||
2. **HTTP API Endpoints** (10 routes) - Fast, efficient search operations on port 37777
|
||||
3. **Worker Service** - Express.js server with FTS5 full-text search
|
||||
4. **SQLite Database** - Persistent storage with FTS5 virtual tables
|
||||
5. **Chroma Vector DB** - Semantic search with hybrid retrieval
|
||||
|
||||
**v5.5.0 Enhancement**: Renamed from "search" to "mem-search" with:
|
||||
- Effectiveness increased from 67% to 100%
|
||||
- Concrete triggers increased from 44% to 85%
|
||||
- 5+ unique identifiers for better scope differentiation
|
||||
- Comprehensive documentation (17 files, 12 operation guides)
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. User Query (Natural Language)
|
||||
@@ -28,10 +34,11 @@ User: "What bugs did we fix last session?"
|
||||
|
||||
### 2. Skill Invocation
|
||||
|
||||
Claude recognizes the intent and invokes the search skill:
|
||||
Claude recognizes the intent and invokes the mem-search skill:
|
||||
- Skill frontmatter (~250 tokens) loaded at session start
|
||||
- Full skill instructions loaded on-demand when skill is invoked
|
||||
- Progressive disclosure pattern minimizes context overhead
|
||||
- "mem-search" naming provides clear scope differentiation from native memory
|
||||
|
||||
### 3. HTTP API Call
|
||||
|
||||
@@ -103,7 +110,7 @@ Claude presents the formatted results naturally in conversation.
|
||||
|
||||
### After: Skill-Based Search
|
||||
|
||||
**Approach**: 1 search skill with progressive disclosure
|
||||
**Approach**: 1 mem-search skill with progressive disclosure
|
||||
|
||||
**Token Cost**: ~250 tokens in skill frontmatter per session
|
||||
- Only skill description loaded at session start
|
||||
@@ -112,7 +119,7 @@ Claude presents the formatted results naturally in conversation.
|
||||
|
||||
**Example Skill Frontmatter**:
|
||||
```markdown
|
||||
# Claude-Mem Search Skill
|
||||
# Claude-Mem mem-search Skill
|
||||
|
||||
Access claude-mem's persistent memory through a comprehensive HTTP API.
|
||||
Search for past work, understand context, and learn from previous decisions.
|
||||
@@ -202,7 +209,7 @@ Returns API documentation in JSON format.
|
||||
|
||||
## Progressive Disclosure Pattern
|
||||
|
||||
The search skill uses progressive disclosure to minimize token usage:
|
||||
The mem-search skill uses progressive disclosure to minimize token usage:
|
||||
|
||||
### Layer 1: Skill Frontmatter (Session Start)
|
||||
|
||||
@@ -212,7 +219,7 @@ The search skill uses progressive disclosure to minimize token usage:
|
||||
|
||||
**Example**:
|
||||
```markdown
|
||||
# Claude-Mem Search Skill
|
||||
# Claude-Mem mem-search Skill
|
||||
|
||||
Access claude-mem's persistent memory through a comprehensive HTTP API.
|
||||
|
||||
@@ -262,10 +269,10 @@ Invoke this skill when users ask about:
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Search Skill Structure
|
||||
### mem-search Skill Structure
|
||||
|
||||
```
|
||||
plugin/skills/search/
|
||||
plugin/skills/mem-search/
|
||||
├── SKILL.md # Main frontmatter (~250 tokens)
|
||||
├── operations/
|
||||
│ ├── observations.md # Search observations
|
||||
@@ -396,7 +403,7 @@ Claude translates to appropriate API call.
|
||||
- MCP configuration removed from `plugin/.mcp.json`
|
||||
|
||||
**New Implementation**: Skill-based search
|
||||
- Skill files: `plugin/skills/search/`
|
||||
- Skill files: `plugin/skills/mem-search/`
|
||||
- HTTP endpoints: `src/services/worker-service.ts` (lines 200-400)
|
||||
- Build script: `npm run build` includes skill files
|
||||
- Sync script: `npm run sync-marketplace` copies to plugin directory
|
||||
@@ -427,11 +434,12 @@ curl "http://localhost:37777/api/search/observations?query=test&limit=1"
|
||||
|
||||
### Skill Not Invoking
|
||||
|
||||
If Claude doesn't invoke the skill:
|
||||
If Claude doesn't invoke the mem-search skill automatically:
|
||||
|
||||
1. Check skill files exist: `ls ~/.claude/plugins/marketplaces/thedotmack/plugin/skills/search/`
|
||||
2. Restart Claude Code session
|
||||
3. Try explicit skill invocation: `/skill search`
|
||||
1. Check skill files exist: `ls ~/.claude/plugins/marketplaces/thedotmack/plugin/skills/mem-search/`
|
||||
2. Restart Claude Code session to reload skill definitions
|
||||
3. Try more explicit phrasing: "Search past sessions for bug fixes" or "What did we do in yesterday's session?"
|
||||
4. Ensure your question is about previous sessions (not current conversation context)
|
||||
|
||||
## Next Steps
|
||||
|
||||
|
Before Width: | Height: | Size: 78 KiB After Width: | Height: | Size: 78 KiB |
|
Before Width: | Height: | Size: 78 KiB After Width: | Height: | Size: 78 KiB |
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 2.1 MiB After Width: | Height: | Size: 2.1 MiB |
@@ -139,15 +139,15 @@ Hooks are configured in `plugin/hooks/hooks.json`:
|
||||
|
||||
### Search Configuration (v5.4.0+)
|
||||
|
||||
**Migration Note**: As of v5.4.0, Claude-Mem uses skill-based search instead of MCP tools.
|
||||
**Migration Note**: As of v5.4.0, Claude-Mem uses skill-based search instead of MCP tools. As of v5.5.0, the skill was renamed to "mem-search" for better scope differentiation.
|
||||
|
||||
**Previous (v5.3.x and earlier)**: MCP search server with 9 tools (~2,500 tokens per session)
|
||||
**Current (v5.4.0+)**: Search skill with HTTP API (~250 tokens per session)
|
||||
**Current (v5.4.0+)**: mem-search skill with HTTP API (~250 tokens per session)
|
||||
|
||||
**No configuration required** - the search skill is automatically available in Claude Code sessions.
|
||||
**No configuration required** - the mem-search skill is automatically available in Claude Code sessions.
|
||||
|
||||
Search operations are now provided via:
|
||||
- **Skill**: `plugin/skills/search/SKILL.md` (auto-invoked when users ask about past work)
|
||||
- **Skill**: `plugin/skills/mem-search/SKILL.md` (auto-invoked when users ask about past work)
|
||||
- **HTTP API**: 10 endpoints on worker service port 37777
|
||||
- **Progressive Disclosure**: Full instructions loaded on-demand only when needed
|
||||
|
||||
@@ -526,11 +526,14 @@ The `release` script:
|
||||
6. Publish to NPM
|
||||
|
||||
```bash
|
||||
# Use the version bump skill (recommended as of v4.3.0)
|
||||
# In Claude Code, run: /skill version-bump
|
||||
# This updates package.json, marketplace.json, and CLAUDE.md
|
||||
# Manual version bump:
|
||||
# 1. Update version in package.json
|
||||
# 2. Update version in plugin/.claude-plugin/plugin.json
|
||||
# 3. Update version at top of CLAUDE.md
|
||||
# 4. Update version badge in README.md
|
||||
# 5. Run: npm run build && npm run sync-marketplace
|
||||
|
||||
# Or manually:
|
||||
# Or use npm version command:
|
||||
npm version 4.3.2
|
||||
|
||||
# Update changelog
|
||||
@@ -68,14 +68,16 @@ Claude Code's hook system provides exactly what we need:
|
||||
|
||||
---
|
||||
|
||||
## The Seven Hook Scripts
|
||||
## The Six Hook Scripts + Pre-Hook
|
||||
|
||||
Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 hooks in sequence.
|
||||
Claude-Mem uses 6 lifecycle hook scripts across 5 lifecycle events, plus 1 pre-hook script for dependency management. SessionStart runs 2 hooks in sequence (after the pre-hook script).
|
||||
|
||||
### Hook 1: SessionStart - Smart Install
|
||||
### Pre-Hook: Smart Install (Before SessionStart)
|
||||
|
||||
**Purpose:** Intelligently manage dependencies and start worker service
|
||||
|
||||
**Note:** This is NOT a lifecycle hook - it's a pre-hook script executed via command chaining before context-hook runs.
|
||||
|
||||
**When:** Claude Code starts (startup, clear, or compact)
|
||||
|
||||
**What it does:**
|
||||
@@ -115,11 +117,11 @@ Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 ho
|
||||
|
||||
---
|
||||
|
||||
### Hook 2: SessionStart - Context Injection
|
||||
### Hook 1: SessionStart - Context Injection
|
||||
|
||||
**Purpose:** Inject relevant context from previous sessions
|
||||
|
||||
**When:** Claude Code starts (runs after smart-install)
|
||||
**When:** Claude Code starts (runs after smart-install pre-hook)
|
||||
|
||||
**What it does:**
|
||||
1. Extracts project name from current working directory
|
||||
@@ -147,14 +149,14 @@ Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 ho
|
||||
|----|------|---|-------|--------|
|
||||
| #2586 | 12:58 AM | 🔵 | Context hook file empty | ~51 |
|
||||
|
||||
*Use claude-mem MCP search to access full details*
|
||||
*Use mem-search skill to access full details*
|
||||
```
|
||||
|
||||
**Source:** `src/hooks/context-hook.ts` → `plugin/scripts/context-hook.js`
|
||||
|
||||
---
|
||||
|
||||
### Hook 3: SessionStart - User Message
|
||||
### Hook 2: SessionStart - User Message
|
||||
|
||||
**Purpose:** Display helpful user messages during first-time setup
|
||||
|
||||
@@ -203,7 +205,7 @@ Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 ho
|
||||
|
||||
---
|
||||
|
||||
### Hook 4: UserPromptSubmit (New Session Hook)
|
||||
### Hook 3: UserPromptSubmit (New Session Hook)
|
||||
|
||||
**Purpose:** Initialize session tracking when user submits a prompt
|
||||
|
||||
@@ -250,7 +252,7 @@ VALUES (?, ?, ?, ...)
|
||||
|
||||
---
|
||||
|
||||
### Hook 5: PostToolUse (Save Observation Hook)
|
||||
### Hook 4: PostToolUse (Save Observation Hook)
|
||||
|
||||
**Purpose:** Capture tool execution observations for later processing
|
||||
|
||||
@@ -311,11 +313,11 @@ VALUES (?, ?, ?, ?, ...)
|
||||
|
||||
---
|
||||
|
||||
### Hook 6: Summary Hook (Mid-Session Checkpoint)
|
||||
### Hook 5: Stop Hook (Summary Generation)
|
||||
|
||||
**Purpose:** Generate AI-powered session summaries during the session
|
||||
|
||||
**When:** Triggered programmatically by the worker service
|
||||
**When:** When Claude stops (triggered by Stop lifecycle event)
|
||||
|
||||
**What it does:**
|
||||
1. Gathers session observations from database
|
||||
@@ -327,7 +329,7 @@ VALUES (?, ?, ?, ?, ...)
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"Summary": [{
|
||||
"Stop": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js"
|
||||
@@ -338,7 +340,7 @@ VALUES (?, ?, ?, ?, ...)
|
||||
```
|
||||
|
||||
**Key decisions:**
|
||||
- ✅ Triggered by worker, not by Claude Code lifecycle
|
||||
- ✅ Triggered by Stop lifecycle event
|
||||
- ✅ Multiple summaries per session (v4.2.0+)
|
||||
- ✅ Summaries are checkpoints, not endings
|
||||
- ✅ Uses Claude Agent SDK for AI compression
|
||||
@@ -366,7 +368,7 @@ VALUES (?, ?, ?, ?, ...)
|
||||
|
||||
---
|
||||
|
||||
### Hook 7: SessionEnd (Cleanup Hook)
|
||||
### Hook 6: SessionEnd (Cleanup Hook)
|
||||
|
||||
**Purpose:** Mark sessions as completed when they end
|
||||
|
||||
@@ -23,7 +23,7 @@ Restart Claude Code. Context from previous sessions will automatically appear in
|
||||
## Key Features
|
||||
|
||||
- 🧠 **Persistent Memory** - Context survives across sessions
|
||||
- 🔍 **Skill-Based Search** - Query your project history with natural language (~2,250 token savings)
|
||||
- 🔍 **mem-search Skill** - Query your project history with natural language (~2,250 token savings)
|
||||
- 🌐 **Web Viewer UI** - Real-time memory stream visualization at http://localhost:37777
|
||||
- 🎨 **Theme Toggle** - Light, dark, and system preference themes
|
||||
- 🤖 **Automatic Operation** - No manual intervention required
|
||||
@@ -55,11 +55,12 @@ Restart Claude Code. Context from previous sessions will automatically appear in
|
||||
```
|
||||
|
||||
**Core Components:**
|
||||
1. **5 Lifecycle Hooks** - SessionStart, UserPromptSubmit, PostToolUse, Stop, SessionEnd
|
||||
2. **Worker Service** - HTTP API on port 37777 managed by PM2
|
||||
3. **SQLite Database** - Stores sessions, observations, summaries with FTS5 search
|
||||
4. **9 MCP Search Tools** - Query historical context with citations
|
||||
5. **Web Viewer UI** - Real-time visualization with SSE and infinite scroll
|
||||
1. **6 Lifecycle Hooks** - SessionStart, UserPromptSubmit, PostToolUse, Stop, SessionEnd, UserMessage
|
||||
2. **Smart Install** - Cached dependency checker (pre-hook script)
|
||||
3. **Worker Service** - HTTP API on port 37777 managed by PM2
|
||||
4. **SQLite Database** - Stores sessions, observations, summaries with FTS5 search
|
||||
5. **mem-search Skill** - Query historical context with natural language
|
||||
6. **Web Viewer UI** - Real-time visualization with SSE and infinite scroll
|
||||
|
||||
See [Architecture Overview](architecture/overview) for details.
|
||||
|
||||
@@ -70,23 +71,23 @@ See [Architecture Overview](architecture/overview) for details.
|
||||
- **PM2**: Process manager (bundled - no global install required)
|
||||
- **SQLite 3**: For persistent storage (bundled)
|
||||
|
||||
## What's New in v5.1.2
|
||||
## What's New in v5.5.1
|
||||
|
||||
**Latest Updates (v5.1.2):**
|
||||
**Latest Updates (v5.5.0):**
|
||||
- mem-search skill enhancement with 100% effectiveness rate
|
||||
- Improved scope differentiation from native conversation memory
|
||||
- Enhanced concrete triggers (85% vs 44% previously)
|
||||
- Comprehensive documentation reorganization (17 total files)
|
||||
|
||||
**Recent Updates (v5.4.0):**
|
||||
- Skill-based search architecture with ~2,250 token savings per session
|
||||
- Progressive disclosure with HTTP API endpoints
|
||||
- Replaced MCP tools with mem-search skill for better efficiency
|
||||
|
||||
**Previous Updates (v5.1.2):**
|
||||
- Theme toggle for light, dark, and system preferences in viewer UI
|
||||
- Improved visual design with theme-aware components
|
||||
|
||||
**Recent Updates (v5.1.0):**
|
||||
- Web-based viewer UI for real-time memory stream visualization
|
||||
- Server-Sent Events (SSE) for instant updates
|
||||
- Infinite scroll pagination with project filtering
|
||||
- 8 new HTTP/SSE endpoints in worker service
|
||||
|
||||
**Previous Updates (v4.3.1):**
|
||||
- Fixed SessionStart hook context injection
|
||||
- Smart install caching for Windows compatibility
|
||||
- Progressive disclosure context with observation timelines
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
@@ -7,13 +7,9 @@ description: "Common issues and solutions for Claude-Mem"
|
||||
|
||||
## Quick Diagnostic Tool
|
||||
|
||||
**NEW:** Use the automated troubleshooting skill for instant diagnosis:
|
||||
Describe any issues you're experiencing to Claude, and the troubleshoot skill will automatically activate to provide diagnosis and fixes.
|
||||
|
||||
```
|
||||
/skill troubleshoot
|
||||
```
|
||||
|
||||
This skill will:
|
||||
The troubleshoot skill will:
|
||||
- ✅ Check PM2 worker status and health
|
||||
- ✅ Verify database existence and integrity
|
||||
- ✅ Test worker service connectivity
|
||||
@@ -21,7 +17,7 @@ This skill will:
|
||||
- ✅ Check port configuration and availability
|
||||
- ✅ Provide automated fixes for common issues
|
||||
|
||||
The skill includes comprehensive diagnostics, automated repair sequences, and detailed troubleshooting workflows for all common issues. Use it before manually troubleshooting below.
|
||||
The skill includes comprehensive diagnostics, automated repair sequences, and detailed troubleshooting workflows for all common issues. Simply describe the problem naturally to invoke it.
|
||||
|
||||
---
|
||||
|
||||
@@ -160,9 +160,9 @@ Context injection uses progressive disclosure for efficient token usage:
|
||||
- Shows full summary details **only if** generated after last observation
|
||||
- Token cost: ~50-200 tokens for index view
|
||||
|
||||
### Layer 2: On-Demand Details (Skill-Based Search)
|
||||
### Layer 2: On-Demand Details (mem-search Skill)
|
||||
- Ask naturally: "What bugs did we fix?" or "How did we implement X?"
|
||||
- Claude auto-invokes search skill to fetch full details
|
||||
- Claude auto-invokes mem-search skill to fetch full details
|
||||
- Search by concept, file, type, or keyword
|
||||
- Timeline context around specific observations
|
||||
- Token cost: ~100-500 tokens per observation fetched
|
||||
@@ -195,7 +195,7 @@ The `/clear` command clears the conversation context visible to Claude AND re-in
|
||||
|
||||
## Searching Your History (v5.4.0+)
|
||||
|
||||
Claude-Mem now uses skill-based search for querying your project history. Simply ask naturally:
|
||||
Claude-Mem uses the mem-search skill for querying your project history. Simply ask naturally:
|
||||
|
||||
```
|
||||
"What bugs did we fix last session?"
|
||||
@@ -204,7 +204,7 @@ Claude-Mem now uses skill-based search for querying your project history. Simply
|
||||
"Show me recent work on this project"
|
||||
```
|
||||
|
||||
Claude automatically recognizes your intent and invokes the search skill, which uses HTTP API endpoints to query your memory efficiently.
|
||||
Claude automatically recognizes your intent and invokes the mem-search skill, which uses HTTP API endpoints to query your memory efficiently.
|
||||
|
||||
**Token Savings**: ~2,250 tokens per session start vs previous MCP approach
|
||||
|
||||
@@ -1,19 +1,21 @@
|
||||
---
|
||||
title: "Skill-Based Search"
|
||||
title: "mem-search Skill"
|
||||
description: "Query your project history with natural language"
|
||||
---
|
||||
|
||||
# Skill-Based Search Usage
|
||||
# mem-search Skill Usage
|
||||
|
||||
Once claude-mem is installed as a plugin, you can search your project history using natural language. Claude automatically invokes the search skill when you ask about past work.
|
||||
Once claude-mem is installed as a plugin, you can search your project history using natural language. Claude automatically invokes the mem-search skill when you ask about past work.
|
||||
|
||||
## How It Works
|
||||
|
||||
**v5.4.0 Migration**: Claude-Mem now uses a skill-based search architecture instead of MCP tools, saving ~2,250 tokens per session start through progressive disclosure.
|
||||
**v5.5.0 Enhancement**: The search skill was renamed to "mem-search" for better scope differentiation, with effectiveness increased from 67% to 100% and enhanced concrete triggers (85% vs 44%).
|
||||
|
||||
**v5.4.0 Architecture**: Claude-Mem uses a skill-based search architecture instead of MCP tools, saving ~2,250 tokens per session start through progressive disclosure.
|
||||
|
||||
**Simple Usage:**
|
||||
- Just ask naturally: *"What did we do last session?"*
|
||||
- Claude recognizes the intent and invokes the search skill
|
||||
- Claude recognizes the intent and invokes the mem-search skill
|
||||
- The skill uses HTTP API endpoints to query your memory
|
||||
- Results are formatted and presented to you
|
||||
|
||||
@@ -22,6 +24,7 @@ Once claude-mem is installed as a plugin, you can search your project history us
|
||||
- **Natural Language**: No need to learn specific tool syntax
|
||||
- **Progressive Disclosure**: Only loads detailed instructions when needed
|
||||
- **Auto-Invoked**: Claude knows when to search based on your questions
|
||||
- **Scope Differentiation**: "mem-search" clearly distinguishes from native conversation memory
|
||||
|
||||
## Quick Reference
|
||||
|
||||
@@ -118,7 +121,7 @@ Once claude-mem is installed as a plugin, you can search your project history us
|
||||
|
||||
## Search Strategy
|
||||
|
||||
The search skill uses a progressive disclosure pattern to efficiently retrieve information:
|
||||
The mem-search skill uses a progressive disclosure pattern to efficiently retrieve information:
|
||||
|
||||
### 1. Ask Naturally
|
||||
|
||||
@@ -127,9 +130,9 @@ Start with a natural language question:
|
||||
"What bugs did we fix related to authentication?"
|
||||
```
|
||||
|
||||
### 2. Claude Invokes Search Skill
|
||||
### 2. Claude Invokes mem-search Skill
|
||||
|
||||
Claude recognizes your intent and loads the search skill (~250 tokens for skill frontmatter).
|
||||
Claude recognizes your intent and loads the mem-search skill (~250 tokens for skill frontmatter).
|
||||
|
||||
### 3. Skill Uses HTTP API
|
||||
|
||||
@@ -197,7 +200,7 @@ You can refine searches using natural language filters:
|
||||
|
||||
## Under the Hood: HTTP API
|
||||
|
||||
The search skill uses HTTP endpoints on the worker service (port 37777):
|
||||
The mem-search skill uses HTTP endpoints on the worker service (port 37777):
|
||||
|
||||
- `GET /api/search/observations` - Full-text search observations
|
||||
- `GET /api/search/sessions` - Full-text search session summaries
|
||||
@@ -366,10 +369,7 @@ npm run worker:restart # Restart if needed
|
||||
npm run worker:logs # View logs
|
||||
```
|
||||
|
||||
Or use the troubleshooting skill:
|
||||
```
|
||||
/skill troubleshoot
|
||||
```
|
||||
Or describe the issue to Claude and the troubleshoot skill will automatically activate to provide diagnosis.
|
||||
|
||||
### Performance Issues
|
||||
|
||||
@@ -383,14 +383,16 @@ If searches seem slow:
|
||||
|
||||
**Architecture Change (v5.4.0)**:
|
||||
- **Before**: 9 MCP tools (~2,500 tokens in tool definitions per session start)
|
||||
- **After**: 1 search skill (~250 tokens in frontmatter, full instructions loaded on-demand)
|
||||
- **After**: 1 mem-search skill (~250 tokens in frontmatter, full instructions loaded on-demand)
|
||||
- **Savings**: ~2,250 tokens per session start
|
||||
- **Migration**: Transparent - users don't need to change how they ask questions
|
||||
|
||||
**v5.5.0 Enhancement**: Renamed from "search" to "mem-search" with improved effectiveness (67% → 100%) and enhanced triggers (44% → 85%).
|
||||
|
||||
**How the Skill Works:**
|
||||
1. User asks a question about past work
|
||||
2. Claude recognizes the intent matches the search skill description
|
||||
3. Skill loads full instructions from `plugin/skills/search/SKILL.md`
|
||||
2. Claude recognizes the intent matches the mem-search skill description
|
||||
3. Skill loads full instructions from `plugin/skills/mem-search/SKILL.md`
|
||||
4. Skill uses `curl` to call HTTP API endpoints
|
||||
5. Results formatted and returned to Claude
|
||||
6. Claude presents results to user
|
||||
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "5.2.2",
|
||||
"version": "5.5.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "claude-mem",
|
||||
"version": "5.2.2",
|
||||
"version": "5.5.1",
|
||||
"license": "AGPL-3.0",
|
||||
"dependencies": {
|
||||
"@anthropic-ai/claude-agent-sdk": "^0.1.27",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "5.4.0",
|
||||
"version": "6.0.0",
|
||||
"description": "Memory compression system for Claude Code - persist context across sessions",
|
||||
"keywords": [
|
||||
"claude",
|
||||
@@ -40,6 +40,7 @@
|
||||
"worker:stop": "pm2 stop claude-mem-worker",
|
||||
"worker:restart": "pm2 restart claude-mem-worker",
|
||||
"worker:logs": "pm2 logs claude-mem-worker",
|
||||
"changelog:generate": "node scripts/generate-changelog.js",
|
||||
"usage:analyze": "node scripts/analyze-usage.js",
|
||||
"usage:today": "node scripts/analyze-usage.js $(date +%Y-%m-%d)"
|
||||
},
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "5.4.0",
|
||||
"version": "5.5.1",
|
||||
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
|
||||
"author": {
|
||||
"name": "Alex Newman"
|
||||
|
||||
@@ -1,3 +1,8 @@
|
||||
{
|
||||
"mcpServers": {}
|
||||
"mcpServers": {
|
||||
"claude-mem-search": {
|
||||
"type": "stdio",
|
||||
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/search-server.mjs"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as I}from"process";import w from"better-sqlite3";import{join as E,dirname as k,basename as W}from"path";import{homedir as O}from"os";import{existsSync as K,mkdirSync as x}from"fs";import{fileURLToPath as U}from"url";function M(){return typeof __dirname<"u"?__dirname:k(U(import.meta.url))}var q=M(),l=process.env.CLAUDE_MEM_DATA_DIR||E(O(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||E(O(),".claude"),J=E(l,"archives"),Q=E(l,"logs"),z=E(l,"trash"),Z=E(l,"backups"),ee=E(l,"settings.json"),f=E(l,"claude-mem.db"),se=E(l,"vector-db"),te=E(R,"settings.json"),re=E(R,"commands"),ne=E(R,"CLAUDE.md");function L(c){x(c,{recursive:!0})}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let m="";n!=null&&(this.level===0&&typeof n=="object"?m=`
|
||||
`+JSON.stringify(n,null,2):m=" "+this.formatData(n));let T="";if(r){let{sessionId:u,sdkSessionId:b,correlationId:p,...a}=r;Object.keys(a).length>0&&(T=` {${Object.entries(a).map(([y,D])=>`${y}=${D}`).join(", ")}}`)}let S=`[${o}] [${i}] [${d}] ${_}${t}${T}${m}`;e===3?console.error(S):console.log(S)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},A=new N;var g=class{db;constructor(){L(l),this.db=new w(f),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as I}from"process";import w from"better-sqlite3";import{join as E,dirname as k,basename as W}from"path";import{homedir as O}from"os";import{existsSync as K,mkdirSync as x}from"fs";import{fileURLToPath as U}from"url";function M(){return typeof __dirname<"u"?__dirname:k(U(import.meta.url))}var q=M(),l=process.env.CLAUDE_MEM_DATA_DIR||E(O(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||E(O(),".claude"),J=E(l,"archives"),Q=E(l,"logs"),z=E(l,"trash"),Z=E(l,"backups"),ee=E(l,"settings.json"),f=E(l,"claude-mem.db"),se=E(l,"vector-db"),te=E(R,"settings.json"),re=E(R,"commands"),ne=E(R,"CLAUDE.md");function L(p){x(p,{recursive:!0})}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
|
||||
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let T="";if(r){let{sessionId:m,sdkSessionId:S,correlationId:c,...a}=r;Object.keys(a).length>0&&(T=` {${Object.entries(a).map(([y,D])=>`${y}=${D}`).join(", ")}}`)}let b=`[${o}] [${i}] [${d}] ${_}${t}${T}${u}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},A=new N;var g=class{db;constructor(){L(l),this.db=new w(f),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -318,23 +318,23 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let m=this.db.prepare(`
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(m.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(u.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let m=this.db.prepare(`
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(m.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(u.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -357,31 +357,31 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE up.id IN (${i})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,_;if(e!==null){let u=`
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,_;if(e!==null){let m=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${o}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let p=this.db.prepare(u).all(e,...i,t+1),a=this.db.prepare(b).all(e,...i,r+1);if(p.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=p.length>0?p[p.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(p){return console.error("[SessionStore] Error getting boundary observations:",p.message),{observations:[],sessions:[],prompts:[]}}}else{let u=`
|
||||
`;try{let c=this.db.prepare(m).all(e,...i,t+1),a=this.db.prepare(S).all(e,...i,r+1);if(c.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=c.length>0?c[c.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary observations:",c.message),{observations:[],sessions:[],prompts:[]}}}else{let m=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let p=this.db.prepare(u).all(s,...i,t),a=this.db.prepare(b).all(s,...i,r+1);if(p.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=p.length>0?p[p.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(p){return console.error("[SessionStore] Error getting boundary timestamps:",p.message),{observations:[],sessions:[],prompts:[]}}}let m=`
|
||||
`;try{let c=this.db.prepare(m).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(c.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=c.length>0?c[c.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary timestamps:",c.message),{observations:[],sessions:[],prompts:[]}}}let u=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
@@ -391,11 +391,11 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,S=`
|
||||
`,b=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let u=this.db.prepare(m).all(d,_,...i),b=this.db.prepare(T).all(d,_,...i),p=this.db.prepare(S).all(d,_,...i);return{observations:u,sessions:b.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:p.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(u){return console.error("[SessionStore] Error querying timeline records:",u.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};import X from"path";import{homedir as F}from"os";import{existsSync as B,readFileSync as j}from"fs";function C(){try{let c=X.join(F(),".claude-mem","settings.json");if(B(c)){let e=JSON.parse(j(c,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function v(c){console.error("[claude-mem cleanup] Hook fired",{input:c?{session_id:c.session_id,cwd:c.cwd,reason:c.reason}:null}),c||(console.log("No input provided - this script is designed to run as a Claude Code SessionEnd hook"),console.log(`
|
||||
Expected input format:`),console.log(JSON.stringify({session_id:"string",cwd:"string",transcript_path:"string",hook_event_name:"SessionEnd",reason:"exit"},null,2)),process.exit(0));let{session_id:e,reason:s}=c;console.error("[claude-mem cleanup] Searching for active SDK session",{session_id:e,reason:s});let t=new g,r=t.findActiveSDKSession(e);r||(console.error("[claude-mem cleanup] No active SDK session found",{session_id:e}),t.close(),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)),console.error("[claude-mem cleanup] Active SDK session found",{session_id:r.id,sdk_session_id:r.sdk_session_id,project:r.project,worker_port:r.worker_port}),t.markSessionCompleted(r.id),console.error("[claude-mem cleanup] Session marked as completed in database"),t.close();try{let n=r.worker_port||C();await fetch(`http://127.0.0.1:${n}/sessions/${r.id}/complete`,{method:"POST",signal:AbortSignal.timeout(1e3)}),console.error("[claude-mem cleanup] Worker notified to stop processing indicator")}catch(n){console.error("[claude-mem cleanup] Failed to notify worker (non-critical):",n)}console.error("[claude-mem cleanup] Cleanup completed successfully"),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)}if(I.isTTY)v(void 0);else{let c="";I.on("data",e=>c+=e),I.on("end",async()=>{let e=c?JSON.parse(c):void 0;await v(e)})}
|
||||
`;try{let m=this.db.prepare(u).all(d,_,...i),S=this.db.prepare(T).all(d,_,...i),c=this.db.prepare(b).all(d,_,...i);return{observations:m,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:c.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(m){return console.error("[SessionStore] Error querying timeline records:",m.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};import X from"path";import{homedir as F}from"os";import{existsSync as B,readFileSync as j}from"fs";function C(){try{let p=X.join(F(),".claude-mem","settings.json");if(B(p)){let e=JSON.parse(j(p,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function v(p){console.error("[claude-mem cleanup] Hook fired",{input:p?{session_id:p.session_id,cwd:p.cwd,reason:p.reason}:null}),p||(console.log("No input provided - this script is designed to run as a Claude Code SessionEnd hook"),console.log(`
|
||||
Expected input format:`),console.log(JSON.stringify({session_id:"string",cwd:"string",transcript_path:"string",hook_event_name:"SessionEnd",reason:"exit"},null,2)),process.exit(0));let{session_id:e,reason:s}=p;console.error("[claude-mem cleanup] Searching for active SDK session",{session_id:e,reason:s});let t=new g,r=t.findActiveSDKSession(e);r||(console.error("[claude-mem cleanup] No active SDK session found",{session_id:e}),t.close(),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)),console.error("[claude-mem cleanup] Active SDK session found",{session_id:r.id,sdk_session_id:r.sdk_session_id,project:r.project,worker_port:r.worker_port}),t.markSessionCompleted(r.id),console.error("[claude-mem cleanup] Session marked as completed in database"),t.close();try{let n=r.worker_port||C();await fetch(`http://127.0.0.1:${n}/sessions/${r.id}/complete`,{method:"POST",signal:AbortSignal.timeout(1e3)}),console.error("[claude-mem cleanup] Worker notified to stop processing indicator")}catch(n){console.error("[claude-mem cleanup] Failed to notify worker (non-critical):",n)}console.error("[claude-mem cleanup] Cleanup completed successfully"),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)}if(I.isTTY)v(void 0);else{let p="";I.on("data",e=>p+=e),I.on("end",async()=>{let e=p?JSON.parse(p):void 0;await v(e)})}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import z from"path";import{stdin as U}from"process";import j from"better-sqlite3";import{join as u,dirname as X,basename as te}from"path";import{homedir as L}from"os";import{existsSync as ie,mkdirSync as F}from"fs";import{fileURLToPath as H}from"url";function P(){return typeof __dirname<"u"?__dirname:X(H(import.meta.url))}var B=P(),m=process.env.CLAUDE_MEM_DATA_DIR||u(L(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||u(L(),".claude"),pe=u(m,"archives"),de=u(m,"logs"),ce=u(m,"trash"),_e=u(m,"backups"),ue=u(m,"settings.json"),A=u(m,"claude-mem.db"),Ee=u(m,"vector-db"),me=u(R,"settings.json"),le=u(R,"commands"),Te=u(R,"CLAUDE.md");function C(a){F(a,{recursive:!0})}function v(){return u(B,"..","..")}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),c="";r?.correlationId?c=`[${r.correlationId}] `:r?.sessionId&&(c=`[session-${r.sessionId}] `);let E="";n!=null&&(this.level===0&&typeof n=="object"?E=`
|
||||
`+JSON.stringify(n,null,2):E=" "+this.formatData(n));let T="";if(r){let{sessionId:l,sdkSessionId:b,correlationId:_,...p}=r;Object.keys(p).length>0&&(T=` {${Object.entries(p).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let S=`[${o}] [${i}] [${d}] ${c}${t}${T}${E}`;e===3?console.error(S):console.log(S)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},y=new N;var g=class{db;constructor(){C(m),this.db=new j(A),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import Y from"path";import{stdin as D}from"process";import F from"better-sqlite3";import{join as E,dirname as U,basename as J}from"path";import{homedir as f}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function X(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=X(),m=process.env.CLAUDE_MEM_DATA_DIR||E(f(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||E(f(),".claude"),re=E(m,"archives"),ne=E(m,"logs"),oe=E(m,"trash"),ie=E(m,"backups"),ae=E(m,"settings.json"),L=E(m,"claude-mem.db"),pe=E(m,"vector-db"),de=E(h,"settings.json"),ce=E(h,"commands"),_e=E(h,"CLAUDE.md");function A(p){M(p,{recursive:!0})}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),d=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let c="";n!=null&&(this.level===0&&typeof n=="object"?c=`
|
||||
`+JSON.stringify(n,null,2):c=" "+this.formatData(n));let l="";if(r){let{sessionId:T,sdkSessionId:b,correlationId:_,...a}=r;Object.keys(a).length>0&&(l=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let S=`[${o}] [${i}] [${d}] ${u}${t}${l}${c}`;e===3?console.error(S):console.log(S)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},C=new O;var g=class{db;constructor(){A(m),this.db=new F(L),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -299,7 +299,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
`).run(s,e).changes===0?(y.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
`).run(s,e).changes===0?(C.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET worker_port = ?
|
||||
WHERE id = ?
|
||||
@@ -318,23 +318,23 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(c.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(c.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -357,7 +357,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE up.id IN (${i})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,c;if(e!==null){let l=`
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,u;if(e!==null){let T=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
@@ -369,7 +369,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE id >= ? ${o}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(l).all(e,...i,t+1),p=this.db.prepare(b).all(e,...i,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let l=`
|
||||
`;try{let _=this.db.prepare(T).all(e,...i,t+1),a=this.db.prepare(b).all(e,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
@@ -381,12 +381,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(l).all(s,...i,t),p=this.db.prepare(b).all(s,...i,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let E=`
|
||||
`;try{let _=this.db.prepare(T).all(s,...i,t),a=this.db.prepare(b).all(s,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let c=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,T=`
|
||||
`,l=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
@@ -397,4 +397,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let l=this.db.prepare(E).all(d,c,...i),b=this.db.prepare(T).all(d,c,...i),_=this.db.prepare(S).all(d,c,...i);return{observations:l,sessions:b.map(p=>({id:p.id,sdk_session_id:p.sdk_session_id,project:p.project,request:p.request,completed:p.completed,next_steps:p.next_steps,created_at:p.created_at,created_at_epoch:p.created_at_epoch})),prompts:_.map(p=>({id:p.id,claude_session_id:p.claude_session_id,project:p.project,prompt:p.prompt_text,created_at:p.created_at,created_at_epoch:p.created_at_epoch}))}}catch(l){return console.error("[SessionStore] Error querying timeline records:",l.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function $(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function D(a,e,s={}){let t=$(a,e,s);return JSON.stringify(t)}import O from"path";import{homedir as W}from"os";import{existsSync as G,readFileSync as Y}from"fs";import{execSync as K}from"child_process";var V=100,q=100,J=1e4;function f(){try{let a=O.join(W(),".claude-mem","settings.json");if(G(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function k(){try{let a=f();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(V)})).ok}catch{return!1}}async function Q(){let a=Date.now();for(;Date.now()-a<J;){if(await k())return!0;await new Promise(e=>setTimeout(e,q))}return!1}async function x(){if(await k())return;let a=v(),e=O.join(a,"node_modules",".bin","pm2"),s=O.join(a,"ecosystem.config.cjs");if(K(`"${e}" start "${s}"`,{cwd:a,stdio:"pipe"}),!await Q())throw new Error("Worker failed to become healthy after restart")}async function Z(a){if(!a)throw new Error("newHook requires input");let{session_id:e,cwd:s,prompt:t}=a,r=z.basename(s);await x();let n=new g,o=n.createSDKSession(e,r,t),i=n.incrementPromptCounter(o);n.saveUserPrompt(e,i,t),console.error(`[new-hook] Session ${o}, prompt #${i}`),n.close();let d=f();try{let c=await fetch(`http://127.0.0.1:${d}/sessions/${o}/init`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({project:r,userPrompt:t}),signal:AbortSignal.timeout(5e3)});if(!c.ok){let E=await c.text();throw new Error(`Failed to initialize session: ${c.status} ${E}`)}}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(D("UserPromptSubmit",!0))}var I="";U.on("data",a=>I+=a);U.on("end",async()=>{let a=I?JSON.parse(I):void 0;await Z(a)});
|
||||
`;try{let T=this.db.prepare(c).all(d,u,...i),b=this.db.prepare(l).all(d,u,...i),_=this.db.prepare(S).all(d,u,...i);return{observations:T,sessions:b.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function P(p,e,s){return p==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:p==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:p==="UserPromptSubmit"||p==="PostToolUse"?{continue:!0,suppressOutput:!0}:p==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function v(p,e,s={}){let t=P(p,e,s);return JSON.stringify(t)}import H from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as $}from"fs";var W=100;function R(){try{let p=H.join(B(),".claude-mem","settings.json");if(j(p)){let e=JSON.parse($(p,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function G(){try{let p=R();return(await fetch(`http://127.0.0.1:${p}/health`,{signal:AbortSignal.timeout(W)})).ok}catch{return!1}}async function y(){if(await G())return;let p=R();throw new Error(`Worker service is not responding on port ${p}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}async function K(p){if(!p)throw new Error("newHook requires input");let{session_id:e,cwd:s,prompt:t}=p,r=Y.basename(s);await y();let n=new g,o=n.createSDKSession(e,r,t),i=n.incrementPromptCounter(o);n.saveUserPrompt(e,i,t),console.error(`[new-hook] Session ${o}, prompt #${i}`),n.close();let d=R(),u=t.startsWith("/")?t.substring(1):t;try{let c=await fetch(`http://127.0.0.1:${d}/sessions/${o}/init`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({project:r,userPrompt:u,promptNumber:i}),signal:AbortSignal.timeout(5e3)});if(!c.ok){let l=await c.text();throw new Error(`Failed to initialize session: ${c.status} ${l}`)}}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(v("UserPromptSubmit",!0))}var I="";D.on("data",p=>I+=p);D.on("end",async()=>{let p=I?JSON.parse(I):void 0;await K(p)});
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as U}from"process";import j from"better-sqlite3";import{join as E,dirname as F,basename as te}from"path";import{homedir as C}from"os";import{existsSync as ie,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function P(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var B=P(),l=process.env.CLAUDE_MEM_DATA_DIR||E(C(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||E(C(),".claude"),de=E(l,"archives"),pe=E(l,"logs"),ce=E(l,"trash"),_e=E(l,"backups"),ue=E(l,"settings.json"),v=E(l,"claude-mem.db"),Ee=E(l,"vector-db"),me=E(h,"settings.json"),le=E(h,"commands"),Te=E(h,"CLAUDE.md");function y(a){X(a,{recursive:!0})}function D(){return E(B,"..","..")}var N=(o=>(o[o.DEBUG=0]="DEBUG",o[o.INFO=1]="INFO",o[o.WARN=2]="WARN",o[o.ERROR=3]="ERROR",o[o.SILENT=4]="SILENT",o))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,o){if(e<this.level)return;let n=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let c="";o!=null&&(this.level===0&&typeof o=="object"?c=`
|
||||
`+JSON.stringify(o,null,2):c=" "+this.formatData(o));let m="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...d}=r;Object.keys(d).length>0&&(m=` {${Object.entries(d).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let g=`[${n}] [${i}] [${p}] ${u}${t}${m}${c}`;e===3?console.error(g):console.log(g)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},b=new O;var R=class{db;constructor(){y(l),this.db=new j(v),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as D}from"process";import X from"better-sqlite3";import{join as m,dirname as U,basename as J}from"path";import{homedir as A}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function F(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=F(),l=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),N=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),re=m(l,"archives"),ne=m(l,"logs"),oe=m(l,"trash"),ie=m(l,"backups"),ae=m(l,"settings.json"),C=m(l,"claude-mem.db"),de=m(l,"vector-db"),pe=m(N,"settings.json"),ce=m(N,"commands"),_e=m(N,"CLAUDE.md");function v(d){M(d,{recursive:!0})}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),I=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=O[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let E="";n!=null&&(this.level===0&&typeof n=="object"?E=`
|
||||
`+JSON.stringify(n,null,2):E=" "+this.formatData(n));let c="";if(r){let{sessionId:T,sdkSessionId:g,correlationId:_,...a}=r;Object.keys(a).length>0&&(c=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let b=`[${o}] [${i}] [${p}] ${u}${t}${c}${E}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},S=new I;var R=class{db;constructor(){v(l),this.db=new X(C),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -244,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id = ?
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",n=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id IN (${i})
|
||||
ORDER BY created_at_epoch ${o}
|
||||
${n}
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
@@ -262,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT files_read, files_modified
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).all(e),r=new Set,o=new Set;for(let n of t){if(n.files_read)try{let i=JSON.parse(n.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(n.files_modified)try{let i=JSON.parse(n.files_modified);Array.isArray(i)&&i.forEach(p=>o.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(o)}}getSessionById(e){return this.db.prepare(`
|
||||
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
@@ -289,17 +289,17 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||1}getPromptCounter(e){return this.db.prepare(`
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,o=r.getTime(),i=this.db.prepare(`
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),i=this.db.prepare(`
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,t,r.toISOString(),o);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
|
||||
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`).get(e).id:i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
`).run(s,e).changes===0?(b.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
`).run(s,e).changes===0?(S.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET worker_port = ?
|
||||
WHERE id = ?
|
||||
@@ -308,33 +308,33 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
LIMIT 1
|
||||
`).get(e)?.worker_port||null}saveUserPrompt(e,s,t){let r=new Date,o=r.getTime();return this.db.prepare(`
|
||||
`).get(e)?.worker_port||null}saveUserPrompt(e,s,t){let r=new Date,n=r.getTime();return this.db.prepare(`
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),o).lastInsertRowid}storeObservation(e,s,t,r){let o=new Date,n=o.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,o.toISOString(),n),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,o.toISOString(),n);return{id:Number(c.lastInsertRowid),createdAtEpoch:n}}storeSummary(e,s,t,r){let o=new Date,n=o.getTime();this.db.prepare(`
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,o.toISOString(),n),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,o.toISOString(),n);return{id:Number(c.lastInsertRowid),createdAtEpoch:n}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -342,12 +342,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",n=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT * FROM session_summaries
|
||||
WHERE id IN (${i})
|
||||
ORDER BY created_at_epoch ${o}
|
||||
${n}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",n=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT
|
||||
up.*,
|
||||
s.project,
|
||||
@@ -355,46 +355,49 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.id IN (${i})
|
||||
ORDER BY up.created_at_epoch ${o}
|
||||
${n}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,o){let n=o?"AND project = ?":"",i=o?[o]:[],p,u;if(e!==null){let T=`
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],p,u;if(e!==null){let T=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${n}
|
||||
WHERE id <= ? ${o}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,S=`
|
||||
`,g=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${n}
|
||||
WHERE id >= ? ${o}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(e,...i,t+1),d=this.db.prepare(S).all(e,...i,r+1);if(_.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
`;try{let _=this.db.prepare(T).all(e,...i,t+1),a=this.db.prepare(g).all(e,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${n}
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,S=`
|
||||
`,g=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${n}
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(s,...i,t),d=this.db.prepare(S).all(s,...i,r+1);if(_.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let c=`
|
||||
`;try{let _=this.db.prepare(T).all(s,...i,t),a=this.db.prepare(g).all(s,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let E=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${n}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,m=`
|
||||
`,c=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${n}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,g=`
|
||||
`,b=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${n.replace("project","s.project")}
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let T=this.db.prepare(c).all(p,u,...i),S=this.db.prepare(m).all(p,u,...i),_=this.db.prepare(g).all(p,u,...i);return{observations:T,sessions:S.map(d=>({id:d.id,sdk_session_id:d.sdk_session_id,project:d.project,request:d.request,completed:d.completed,next_steps:d.next_steps,created_at:d.created_at,created_at_epoch:d.created_at_epoch})),prompts:_.map(d=>({id:d.id,claude_session_id:d.claude_session_id,project:d.project,prompt:d.prompt_text,created_at:d.created_at,created_at_epoch:d.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function $(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function f(a,e,s={}){let t=$(a,e,s);return JSON.stringify(t)}import I from"path";import{homedir as W}from"os";import{existsSync as G,readFileSync as Y}from"fs";import{execSync as K}from"child_process";var V=100,q=100,J=1e4;function L(){try{let a=I.join(W(),".claude-mem","settings.json");if(G(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function k(){try{let a=L();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(V)})).ok}catch{return!1}}async function Q(){let a=Date.now();for(;Date.now()-a<J;){if(await k())return!0;await new Promise(e=>setTimeout(e,q))}return!1}async function x(){if(await k())return;let a=D(),e=I.join(a,"node_modules",".bin","pm2"),s=I.join(a,"ecosystem.config.cjs");if(K(`"${e}" start "${s}"`,{cwd:a,stdio:"pipe"}),!await Q())throw new Error("Worker failed to become healthy after restart")}var z=new Set(["ListMcpResourcesTool"]);async function Z(a){if(!a)throw new Error("saveHook requires input");let{session_id:e,tool_name:s,tool_input:t,tool_response:r}=a;if(z.has(s)){console.log(f("PostToolUse",!0));return}await x();let o=new R,n=o.createSDKSession(e,"",""),i=o.getPromptCounter(n);o.close();let p=b.formatTool(s,t),u=L();b.dataIn("HOOK",`PostToolUse: ${p}`,{sessionId:n,workerPort:u});try{let c=await fetch(`http://127.0.0.1:${u}/sessions/${n}/observations`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({tool_name:s,tool_input:t!==void 0?JSON.stringify(t):"{}",tool_response:r!==void 0?JSON.stringify(r):"{}",prompt_number:i}),signal:AbortSignal.timeout(2e3)});if(!c.ok){let m=await c.text();throw b.failure("HOOK","Failed to send observation",{sessionId:n,status:c.status},m),new Error(`Failed to send observation to worker: ${c.status} ${m}`)}b.debug("HOOK","Observation sent successfully",{sessionId:n,toolName:s})}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(f("PostToolUse",!0))}var A="";U.on("data",a=>A+=a);U.on("end",async()=>{let a=A?JSON.parse(A):void 0;await Z(a)});
|
||||
`;try{let T=this.db.prepare(E).all(p,u,...i),g=this.db.prepare(c).all(p,u,...i),_=this.db.prepare(b).all(p,u,...i);return{observations:T,sessions:g.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function H(d,e,s){return d==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:d==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:d==="UserPromptSubmit"||d==="PostToolUse"?{continue:!0,suppressOutput:!0}:d==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function f(d,e,s={}){let t=H(d,e,s);return JSON.stringify(t)}import P from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as $}from"fs";var W=100;function h(){try{let d=P.join(B(),".claude-mem","settings.json");if(j(d)){let e=JSON.parse($(d,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function G(){try{let d=h();return(await fetch(`http://127.0.0.1:${d}/health`,{signal:AbortSignal.timeout(W)})).ok}catch{return!1}}async function y(){if(await G())return;let d=h();throw new Error(`Worker service is not responding on port ${d}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}var Y=new Set(["ListMcpResourcesTool","SlashCommand","Skill","TodoWrite","AskUserQuestion"]);async function K(d){if(!d)throw new Error("saveHook requires input");let{session_id:e,cwd:s,tool_name:t,tool_input:r,tool_response:n}=d;if(Y.has(t)){console.log(f("PostToolUse",!0));return}await y();let o=new R,i=o.createSDKSession(e,"",""),p=o.getPromptCounter(i);o.close();let u=S.formatTool(t,r),E=h();S.dataIn("HOOK",`PostToolUse: ${u}`,{sessionId:i,workerPort:E});try{let c=await fetch(`http://127.0.0.1:${E}/sessions/${i}/observations`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({tool_name:t,tool_input:r!==void 0?JSON.stringify(r):"{}",tool_response:n!==void 0?JSON.stringify(n):"{}",prompt_number:p,cwd:s||""}),signal:AbortSignal.timeout(2e3)});if(!c.ok){let b=await c.text();throw S.failure("HOOK","Failed to send observation",{sessionId:i,status:c.status},b),new Error(`Failed to send observation to worker: ${c.status} ${b}`)}S.debug("HOOK","Observation sent successfully",{sessionId:i,toolName:t})}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(f("PostToolUse",!0))}var L="";D.on("data",d=>L+=d);D.on("end",async()=>{let d=L?JSON.parse(L):void 0;await K(d)});
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as U}from"process";import j from"better-sqlite3";import{join as _,dirname as F,basename as se}from"path";import{homedir as A}from"os";import{existsSync as oe,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function P(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var B=P(),E=process.env.CLAUDE_MEM_DATA_DIR||_(A(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||_(A(),".claude"),ae=_(E,"archives"),de=_(E,"logs"),pe=_(E,"trash"),ce=_(E,"backups"),_e=_(E,"settings.json"),C=_(E,"claude-mem.db"),ue=_(E,"vector-db"),me=_(h,"settings.json"),Ee=_(h,"commands"),le=_(h,"CLAUDE.md");function y(a){X(a,{recursive:!0})}function v(){return _(B,"..","..")}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let m="";n!=null&&(this.level===0&&typeof n=="object"?m=`
|
||||
`+JSON.stringify(n,null,2):m=" "+this.formatData(n));let T="";if(r){let{sessionId:l,sdkSessionId:b,correlationId:c,...d}=r;Object.keys(d).length>0&&(T=` {${Object.entries(d).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let g=`[${o}] [${i}] [${p}] ${u}${t}${T}${m}`;e===3?console.error(g):console.log(g)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},S=new O;var R=class{db;constructor(){y(E),this.db=new j(C),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as k}from"process";import{readFileSync as x,existsSync as U}from"fs";import B from"better-sqlite3";import{join as m,dirname as F,basename as ne}from"path";import{homedir as A}from"os";import{existsSync as de,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function j(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var ce=j(),E=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),f=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),ue=m(E,"archives"),_e=m(E,"logs"),me=m(E,"trash"),le=m(E,"backups"),Ee=m(E,"settings.json"),y=m(E,"claude-mem.db"),Te=m(E,"vector-db"),ge=m(f,"settings.json"),Se=m(f,"commands"),be=m(f,"CLAUDE.md");function C(i){X(i,{recursive:!0})}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let a=new Date().toISOString().replace("T"," ").substring(0,23),o=O[e].padEnd(5),d=s.padEnd(6),c="";r?.correlationId?c=`[${r.correlationId}] `:r?.sessionId&&(c=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
|
||||
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let l="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...p}=r;Object.keys(p).length>0&&(l=` {${Object.entries(p).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let b=`[${a}] [${o}] [${d}] ${c}${t}${l}${u}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},g=new N;var R=class{db;constructor(){C(E),this.db=new B(y),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -63,7 +63,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(sdk_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
|
||||
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(p=>p.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
|
||||
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(d=>d.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
|
||||
CREATE TABLE session_summaries_new (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
sdk_session_id TEXT NOT NULL,
|
||||
@@ -244,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id = ?
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id IN (${i})
|
||||
WHERE id IN (${o})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
${a}
|
||||
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
@@ -262,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT files_read, files_modified
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
`).all(e),r=new Set,n=new Set;for(let a of t){if(a.files_read)try{let o=JSON.parse(a.files_read);Array.isArray(o)&&o.forEach(d=>r.add(d))}catch{}if(a.files_modified)try{let o=JSON.parse(a.files_modified);Array.isArray(o)&&o.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
@@ -289,17 +289,17 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||1}getPromptCounter(e){return this.db.prepare(`
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),i=this.db.prepare(`
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),o=this.db.prepare(`
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
|
||||
`).run(e,e,s,t,r.toISOString(),n);return o.lastInsertRowid===0||o.changes===0?this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`).get(e).id:i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
`).get(e).id:o.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
`).run(s,e).changes===0?(S.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
`).run(s,e).changes===0?(g.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET worker_port = ?
|
||||
WHERE id = ?
|
||||
@@ -312,29 +312,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,a=n.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let m=this.db.prepare(`
|
||||
`).run(e,e,s,n.toISOString(),a),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(m.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),a);return{id:Number(u.lastInsertRowid),createdAtEpoch:a}}storeSummary(e,s,t,r){let n=new Date,a=n.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let m=this.db.prepare(`
|
||||
`).run(e,e,s,n.toISOString(),a),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(m.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),a);return{id:Number(u.lastInsertRowid),createdAtEpoch:a}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -342,59 +342,77 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT * FROM session_summaries
|
||||
WHERE id IN (${i})
|
||||
WHERE id IN (${o})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
${a}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT
|
||||
up.*,
|
||||
s.project,
|
||||
s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.id IN (${i})
|
||||
WHERE up.id IN (${o})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],p,u;if(e!==null){let l=`
|
||||
${a}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let a=n?"AND project = ?":"",o=n?[n]:[],d,c;if(e!==null){let T=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
WHERE id <= ? ${a}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${o}
|
||||
WHERE id >= ? ${a}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let c=this.db.prepare(l).all(e,...i,t+1),d=this.db.prepare(b).all(e,...i,r+1);if(c.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=c.length>0?c[c.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary observations:",c.message),{observations:[],sessions:[],prompts:[]}}}else{let l=`
|
||||
`;try{let _=this.db.prepare(T).all(e,...o,t+1),p=this.db.prepare(S).all(e,...o,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch <= ? ${a}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
WHERE created_at_epoch >= ? ${a}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let c=this.db.prepare(l).all(s,...i,t),d=this.db.prepare(b).all(s,...i,r+1);if(c.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=c.length>0?c[c.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary timestamps:",c.message),{observations:[],sessions:[],prompts:[]}}}let m=`
|
||||
`;try{let _=this.db.prepare(T).all(s,...o,t),p=this.db.prepare(S).all(s,...o,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let u=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${a}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,T=`
|
||||
`,l=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${a}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,g=`
|
||||
`,b=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${a.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let l=this.db.prepare(m).all(p,u,...i),b=this.db.prepare(T).all(p,u,...i),c=this.db.prepare(g).all(p,u,...i);return{observations:l,sessions:b.map(d=>({id:d.id,sdk_session_id:d.sdk_session_id,project:d.project,request:d.request,completed:d.completed,next_steps:d.next_steps,created_at:d.created_at,created_at_epoch:d.created_at_epoch})),prompts:c.map(d=>({id:d.id,claude_session_id:d.claude_session_id,project:d.project,prompt:d.prompt_text,created_at:d.created_at,created_at_epoch:d.created_at_epoch}))}}catch(l){return console.error("[SessionStore] Error querying timeline records:",l.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function $(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function D(a,e,s={}){let t=$(a,e,s);return JSON.stringify(t)}import f from"path";import{homedir as W}from"os";import{existsSync as G,readFileSync as Y}from"fs";import{execSync as K}from"child_process";var q=100,V=100,J=1e4;function I(){try{let a=f.join(W(),".claude-mem","settings.json");if(G(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function k(){try{let a=I();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(q)})).ok}catch{return!1}}async function Q(){let a=Date.now();for(;Date.now()-a<J;){if(await k())return!0;await new Promise(e=>setTimeout(e,V))}return!1}async function x(){if(await k())return;let a=v(),e=f.join(a,"node_modules",".bin","pm2"),s=f.join(a,"ecosystem.config.cjs");if(K(`"${e}" start "${s}"`,{cwd:a,stdio:"pipe"}),!await Q())throw new Error("Worker failed to become healthy after restart")}async function z(a){if(!a)throw new Error("summaryHook requires input");let{session_id:e}=a;await x();let s=new R,t=s.createSDKSession(e,"",""),r=s.getPromptCounter(t);s.close();let n=I();S.dataIn("HOOK","Stop: Requesting summary",{sessionId:t,workerPort:n,promptNumber:r});try{let o=await fetch(`http://127.0.0.1:${n}/sessions/${t}/summarize`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({prompt_number:r}),signal:AbortSignal.timeout(2e3)});if(!o.ok){let i=await o.text();throw S.failure("HOOK","Failed to generate summary",{sessionId:t,status:o.status},i),new Error(`Failed to request summary from worker: ${o.status} ${i}`)}S.debug("HOOK","Summary request sent successfully",{sessionId:t})}catch(o){throw o.cause?.code==="ECONNREFUSED"||o.name==="TimeoutError"||o.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):o}finally{await fetch(`http://127.0.0.1:${n}/api/processing`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({isProcessing:!1})})}console.log(D("Stop",!0))}var L="";U.on("data",a=>L+=a);U.on("end",async()=>{let a=L?JSON.parse(L):void 0;await z(a)});
|
||||
`;try{let T=this.db.prepare(u).all(d,c,...o),S=this.db.prepare(l).all(d,c,...o),_=this.db.prepare(b).all(d,c,...o);return{observations:T,sessions:S.map(p=>({id:p.id,sdk_session_id:p.sdk_session_id,project:p.project,request:p.request,completed:p.completed,next_steps:p.next_steps,created_at:p.created_at,created_at_epoch:p.created_at_epoch})),prompts:_.map(p=>({id:p.id,claude_session_id:p.claude_session_id,project:p.project,prompt:p.prompt_text,created_at:p.created_at,created_at_epoch:p.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function P(i,e,s){return i==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:i==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:i==="UserPromptSubmit"||i==="PostToolUse"?{continue:!0,suppressOutput:!0}:i==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function v(i,e,s={}){let t=P(i,e,s);return JSON.stringify(t)}import $ from"path";import{homedir as W}from"os";import{existsSync as G,readFileSync as Y}from"fs";var K=100;function h(){try{let i=$.join(W(),".claude-mem","settings.json");if(G(i)){let e=JSON.parse(Y(i,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function q(){try{let i=h();return(await fetch(`http://127.0.0.1:${i}/health`,{signal:AbortSignal.timeout(K)})).ok}catch{return!1}}async function D(){if(await q())return;let i=h();throw new Error(`Worker service is not responding on port ${i}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}import{appendFileSync as V}from"fs";import{homedir as J}from"os";import{join as Q}from"path";var z=Q(J(),".claude-mem","silent.log");function I(i,e,s=""){let t=new Date().toISOString(),o=((new Error().stack||"").split(`
|
||||
`)[2]||"").match(/at\s+(?:.*\s+)?\(?([^:]+):(\d+):(\d+)\)?/),d=o?`${o[1].split("/").pop()}:${o[2]}`:"unknown",c=`[${t}] [${d}] ${i}`;if(e!==void 0)try{c+=` ${JSON.stringify(e)}`}catch(u){c+=` [stringify error: ${u}]`}c+=`
|
||||
`;try{V(z,c)}catch(u){console.error("[silent-debug] Failed to write to log:",u)}return s}function Z(i){if(!i||!U(i))return"";try{let e=x(i,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="user"&&r.message?.content){let n=r.message.content;if(typeof n=="string")return n;if(Array.isArray(n))return n.filter(o=>o.type==="text").map(o=>o.text).join(`
|
||||
`)}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:i},e)}return""}function ee(i){if(!i||!U(i))return"";try{let e=x(i,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="assistant"&&r.message?.content){let n="",a=r.message.content;return typeof a=="string"?n=a:Array.isArray(a)&&(n=a.filter(d=>d.type==="text").map(d=>d.text).join(`
|
||||
`)),n=n.replace(/<system-reminder>[\s\S]*?<\/system-reminder>/g,""),n=n.replace(/\n{3,}/g,`
|
||||
|
||||
`).trim(),n}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:i},e)}return""}async function se(i){if(!i)throw new Error("summaryHook requires input");let{session_id:e}=i;await D();let s=new R,t=s.createSDKSession(e,"",""),r=s.getPromptCounter(t),n=s.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project
|
||||
FROM sdk_sessions WHERE id = ?
|
||||
`).get(t),a=s.db.prepare(`
|
||||
SELECT COUNT(*) as count
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).get(n?.sdk_session_id);I("[summary-hook] Session diagnostics",{claudeSessionId:e,sessionDbId:t,sdkSessionId:n?.sdk_session_id,project:n?.project,promptNumber:r,observationCount:a?.count||0,transcriptPath:i.transcript_path}),s.close();let o=h(),d=Z(i.transcript_path||""),c=ee(i.transcript_path||"");I("[summary-hook] Extracted messages",{hasLastUserMessage:!!d,hasLastAssistantMessage:!!c,lastAssistantPreview:c.substring(0,200),lastAssistantLength:c.length}),g.dataIn("HOOK","Stop: Requesting summary",{sessionId:t,workerPort:o,promptNumber:r,hasLastUserMessage:!!d,hasLastAssistantMessage:!!c});try{let u=await fetch(`http://127.0.0.1:${o}/sessions/${t}/summarize`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({prompt_number:r,last_user_message:d,last_assistant_message:c}),signal:AbortSignal.timeout(2e3)});if(!u.ok){let l=await u.text();throw g.failure("HOOK","Failed to generate summary",{sessionId:t,status:u.status},l),new Error(`Failed to request summary from worker: ${u.status} ${l}`)}g.debug("HOOK","Summary request sent successfully",{sessionId:t})}catch(u){throw u.cause?.code==="ECONNREFUSED"||u.name==="TimeoutError"||u.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):u}finally{await fetch(`http://127.0.0.1:${o}/api/processing`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({isProcessing:!1})})}console.log(v("Stop",!0))}var L="";k.on("data",i=>L+=i);k.on("end",async()=>{let i=L?JSON.parse(L):void 0;await se(i)});
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env node
|
||||
import{execSync as _}from"child_process";import{join as i}from"path";import{homedir as p}from"os";import{existsSync as D}from"fs";import l from"path";import{homedir as f}from"os";import{existsSync as g,readFileSync as h}from"fs";import{join as e,dirname as m,basename as A}from"path";import{homedir as c}from"os";import{fileURLToPath as u}from"url";function d(){return typeof __dirname<"u"?__dirname:m(u(import.meta.url))}var w=d(),t=process.env.CLAUDE_MEM_DATA_DIR||e(c(),".claude-mem"),s=process.env.CLAUDE_CONFIG_DIR||e(c(),".claude"),P=e(t,"archives"),R=e(t,"logs"),S=e(t,"trash"),I=e(t,"backups"),b=e(t,"settings.json"),v=e(t,"claude-mem.db"),H=e(t,"vector-db"),L=e(s,"settings.json"),M=e(s,"commands"),U=e(s,"CLAUDE.md");function a(){try{let o=l.join(f(),".claude-mem","settings.json");if(g(o)){let n=JSON.parse(h(o,"utf-8")),r=parseInt(n.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(r))return r}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}var x=i(p(),".claude","plugins","marketplaces","thedotmack"),k=i(x,"node_modules");D(k)||(console.error(`
|
||||
import{execSync as p}from"child_process";import{join as r}from"path";import{homedir as s}from"os";import{existsSync as u}from"fs";import i from"path";import{homedir as a}from"os";import{existsSync as c,readFileSync as l}from"fs";function n(){try{let e=i.join(a(),".claude-mem","settings.json");if(c(e)){let t=JSON.parse(l(e,"utf-8")),o=parseInt(t.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(o))return o}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}var m=r(s(),".claude","plugins","marketplaces","thedotmack"),d=r(m,"node_modules");u(d)||(console.error(`
|
||||
---
|
||||
\u{1F389} Note: This appears under Plugin Hook Error, but it's not an error. That's the only option for
|
||||
user messages in Claude Code UI until a better method is provided.
|
||||
@@ -17,12 +17,12 @@ Dependencies have been installed in the background. This only happens once.
|
||||
Thank you for installing Claude-Mem!
|
||||
|
||||
This message was not added to your startup context, so you can continue working as normal.
|
||||
`),process.exit(3));try{let o=i(p(),".claude","plugins","marketplaces","thedotmack","plugin","scripts","context-hook.js"),n=_(`node "${o}" --colors`,{encoding:"utf8"}),r=a();console.error(`
|
||||
`),process.exit(3));try{let e=r(s(),".claude","plugins","marketplaces","thedotmack","plugin","scripts","context-hook.js"),t=p(`node "${e}" --colors`,{encoding:"utf8"}),o=n();console.error(`
|
||||
|
||||
\u{1F4DD} Claude-Mem Context Loaded
|
||||
\u2139\uFE0F Note: This appears as stderr but is informational only
|
||||
|
||||
`+n+`
|
||||
`+t+`
|
||||
|
||||
\u{1F4FA} Watch live in browser http://localhost:${r}/ (New! v5.1)
|
||||
`)}catch(o){console.error(`\u274C Failed to load context display: ${o}`)}process.exit(3);
|
||||
\u{1F4FA} Watch live in browser http://localhost:${o}/ (New! v5.1)
|
||||
`)}catch(e){console.error(`\u274C Failed to load context display: ${e}`)}process.exit(3);
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
---
|
||||
name: mem-search
|
||||
description: Search claude-mem's persistent cross-session memory database to find work from previous conversations days, weeks, or months ago. Access past session summaries, bug fixes, feature implementations, and decisions that are NOT in the current conversation context. Use when user asks "did we already solve this?", "how did we do X last time?", "what happened in last week's session?", or needs information from previous sessions stored in the PM2-managed database. Searches observations, session summaries, and user prompts across entire project history.
|
||||
---
|
||||
|
||||
# Memory Search
|
||||
|
||||
Access claude-mem's persistent cross-session memory through HTTP API. Find past work, understand context across sessions, and learn from previous decisions.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
**Use when users ask about work from PREVIOUS sessions** (not current conversation):
|
||||
|
||||
### Temporal Triggers (Key Indicators)
|
||||
- "Did we **already** fix this bug?" or "Have we seen this error **before**?"
|
||||
- "How did we solve X **last time**?" or "What approach did we take **previously**?"
|
||||
- "What did we do in **yesterday's/last week's/last month's** session?"
|
||||
- "**When** did we last work on this?" or "What's the **history** of this file?"
|
||||
|
||||
### Cross-Session Queries
|
||||
- "Show me all authentication-related changes **across all sessions**"
|
||||
- "What features did we add **last month**?" (not "today" or "this session")
|
||||
- "Why did we choose this approach **before**?" (decisions from past sessions)
|
||||
- "What files did we modify **when we added X**?" (historical context)
|
||||
|
||||
**Do NOT use** for current session work, future planning, or questions Claude can answer from current conversation context.
|
||||
|
||||
## Common Trigger Phrases
|
||||
|
||||
This skill activates when detecting phrases about **cross-session history**:
|
||||
|
||||
- "Did we already solve this?" / "Have we done this before?"
|
||||
- "How did we implement X last time?"
|
||||
- "What did we work on yesterday/last week/last month?"
|
||||
- "Show me the history of [file/feature/decision]"
|
||||
- "When did we fix/add/change X?"
|
||||
- "What was happening around [date/time]?"
|
||||
- "Catch me up on recent sessions" / "What have we been doing?"
|
||||
- "What changes to [filename] across all sessions?"
|
||||
|
||||
**Unique identifiers:** claude-mem, persistent memory, cross-session database, session history, PM2-managed database
|
||||
|
||||
## Available Operations
|
||||
|
||||
### Full-Text Search
|
||||
- **observations** - Search all observations by keyword (bugs, features, decisions, discoveries, changes)
|
||||
- Use when: "How did we implement X?" or "What bugs did we fix?"
|
||||
- Example: Search for "authentication JWT" to find auth-related work
|
||||
|
||||
- **sessions** - Search session summaries to find what was accomplished when
|
||||
- Use when: "What did we accomplish last time?" or "What was the goal of that session?"
|
||||
- Example: Find sessions where "added login feature"
|
||||
|
||||
- **prompts** - Find what users have asked about in previous sessions
|
||||
- Use when: "Did I ask about this before?" or "What did I request last week?"
|
||||
- Example: Search for "database migration" in past user prompts
|
||||
|
||||
### Filtered Search
|
||||
- **by-type** - Filter observations by type (bugfix, feature, refactor, decision, discovery, change)
|
||||
- Use when: "Show me all bug fixes" or "List features we added"
|
||||
- Example: Get all observations with type=bugfix from last month
|
||||
|
||||
- **by-concept** - Find observations tagged with specific concepts (problem-solution, how-it-works, gotcha)
|
||||
- Use when: "What patterns did we discover?" or "Show me gotchas"
|
||||
- Example: Find all observations tagged with concept "gotcha"
|
||||
|
||||
- **by-file** - Find all work related to a specific file path across all sessions
|
||||
- Use when: "What changes to auth.ts?" or "History of this file"
|
||||
- Example: Get all work related to "src/auth/login.ts"
|
||||
|
||||
### Timeline & Context
|
||||
- **recent-context** - Get last N sessions with summaries and observations
|
||||
- Use when: "What's been happening?" or "Catch me up on recent work"
|
||||
- Example: Get last 3 sessions with limit=3
|
||||
|
||||
- **timeline** - Get chronological context around a specific point in time (before/after window)
|
||||
- Use when: "What was happening around [date]?" or "Show me context from that time"
|
||||
- Example: Timeline around session 123 with depth 5 before and after
|
||||
|
||||
- **timeline-by-query** - Search first, then get timeline context around best match
|
||||
- Use when: "When did we implement auth?" combined with "show me context around that time"
|
||||
- Example: Search for "OAuth implementation" then get surrounding timeline
|
||||
|
||||
For detailed instructions on any operation, read the corresponding file in [operations/](operations/).
|
||||
|
||||
## Quick Decision Guide
|
||||
|
||||
**What is the user asking about?**
|
||||
|
||||
1. **Recent work** (last 3-5 sessions) → Use **recent-context** with limit=3-5
|
||||
2. **Specific topic/keyword** (bugs, features, decisions) → Use **observations** search
|
||||
3. **Specific file history** (changes to a file) → Use **by-file** search
|
||||
4. **Timeline/chronology** (what happened when) → Use **timeline** or **timeline-by-query**
|
||||
5. **Type-specific** (all bug fixes, all features) → Use **by-type** filter
|
||||
|
||||
**Most common:** Use **observations** search for general "how did we..." questions.
|
||||
|
||||
## Progressive Disclosure Workflow (Token Efficiency)
|
||||
|
||||
**Core Principle**: Find high-signal items in index format FIRST (~50-100 tokens each), then request full details ONLY for relevant items (~500-1000 tokens each).
|
||||
|
||||
**4-Step Workflow:**
|
||||
|
||||
1. **Start with Index Format**
|
||||
- Always use `format=index` initially
|
||||
- Set `limit=3-5` (not 10-20)
|
||||
- Review titles and dates to assess relevance
|
||||
- Token cost: ~50-100 per result
|
||||
|
||||
2. **Identify Relevant Items**
|
||||
- Scan index results
|
||||
- Discard irrelevant items from list
|
||||
- Keep only 1-3 most relevant
|
||||
|
||||
3. **Request Full Details Selectively**
|
||||
- Use `format=full` ONLY for specific relevant items
|
||||
- Token cost: ~500-1000 per result
|
||||
- **10x cost difference** - be selective
|
||||
|
||||
4. **Refine with Filters**
|
||||
- Use type, dateRange, concepts, files filters
|
||||
- Paginate with offset if needed
|
||||
- Narrow scope before expanding limits
|
||||
|
||||
**DO:**
|
||||
- ✅ Start with `format=index` and `limit=3-5`
|
||||
- ✅ Use filters (type, dateRange, concepts, files) to narrow results
|
||||
- ✅ Request `format=full` ONLY for specific relevant items
|
||||
- ✅ Use offset for pagination instead of large limits
|
||||
|
||||
**DON'T:**
|
||||
- ❌ Jump straight to `format=full`
|
||||
- ❌ Request `limit=20` without assessing index results first
|
||||
- ❌ Load full details for all results upfront
|
||||
- ❌ Skip index format to "save a step" (costs 10x more tokens)
|
||||
|
||||
See [principles/progressive-disclosure.md](principles/progressive-disclosure.md) for complete workflow with examples and token calculations.
|
||||
|
||||
## Quick Reference Table
|
||||
|
||||
| Need | Operation | Key Parameters |
|
||||
|------|-----------|----------------|
|
||||
| Recent context | recent-context | limit=3-5 |
|
||||
| Search observations | observations | query, format=index, limit=5 |
|
||||
| Search sessions | sessions | query, format=index, limit=5 |
|
||||
| Find by type | by-type | type=(bugfix\|feature\|decision), format=index |
|
||||
| Find by file | by-file | filePath, format=index |
|
||||
| Timeline around event | timeline | anchor=(sessionDbId), depth_before=5, depth_after=5 |
|
||||
| Search + timeline | timeline-by-query | query, mode=auto |
|
||||
|
||||
## Common Workflows
|
||||
|
||||
For step-by-step guides on typical user requests, see [operations/common-workflows.md](operations/common-workflows.md):
|
||||
- Understanding past work from previous sessions
|
||||
- Finding specific bug fixes from history
|
||||
- Understanding file history across sessions
|
||||
- Timeline investigation workflows
|
||||
- Search composition patterns
|
||||
|
||||
## Response Formatting
|
||||
|
||||
For guidelines on presenting search results to users, see [operations/formatting.md](operations/formatting.md):
|
||||
- Index format responses (compact lists with titles/dates)
|
||||
- Full format responses (complete observation details)
|
||||
- Timeline responses (chronologically grouped)
|
||||
- Error handling and user-friendly messages
|
||||
|
||||
## Technical Notes
|
||||
|
||||
- **Port:** Default 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
|
||||
- **Response format:** Always JSON
|
||||
- **Search engine:** FTS5 full-text search + structured filters
|
||||
- **All operations:** HTTP GET with query parameters
|
||||
- **Worker:** PM2-managed background process
|
||||
|
||||
## Error Handling
|
||||
|
||||
If HTTP request fails:
|
||||
1. Inform user the claude-mem search service isn't available
|
||||
2. Suggest checking if worker is running: `pm2 list` or `pm2 status claude-mem-worker`
|
||||
3. Offer to help troubleshoot using the troubleshoot skill
|
||||
|
||||
## Resources
|
||||
|
||||
**Principles:**
|
||||
- [principles/progressive-disclosure.md](principles/progressive-disclosure.md) - Complete 4-step workflow with token calculations
|
||||
- [principles/anti-patterns.md](principles/anti-patterns.md) - 5 anti-patterns to avoid with LLM behavior insights
|
||||
|
||||
**Operations:**
|
||||
- [operations/](operations/) - Detailed instructions for each operation (9 operations + help)
|
||||
- [operations/common-workflows.md](operations/common-workflows.md) - Step-by-step workflow guides
|
||||
- [operations/formatting.md](operations/formatting.md) - Response formatting templates
|
||||
|
||||
---
|
||||
|
||||
**Remember:** This skill searches **cross-session persistent memory**, NOT current conversation. Start with index format for token efficiency. Use temporal triggers to differentiate from native Claude memory.
|
||||
@@ -0,0 +1,124 @@
|
||||
# Search by Concept
|
||||
|
||||
Find observations tagged with specific concepts.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What discoveries did we make?"
|
||||
- User asks: "What patterns did we identify?"
|
||||
- User asks: "What gotchas did we encounter?"
|
||||
- Looking for observations with semantic tags
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-concept?concept=discovery&format=index&limit=5"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- **concept** (required): Concept tag to search for
|
||||
- `discovery` - New discoveries and insights
|
||||
- `problem-solution` - Problems and their solutions
|
||||
- `what-changed` - Change descriptions
|
||||
- `how-it-works` - Explanations of mechanisms
|
||||
- `pattern` - Identified patterns
|
||||
- `gotcha` - Edge cases and gotchas
|
||||
- `change` - General changes
|
||||
- **format**: "index" (summary) or "full" (complete details). Default: "full"
|
||||
- **limit**: Number of results (default: 20, max: 100)
|
||||
- **project**: Filter by project name (optional)
|
||||
- **dateRange**: Filter by date range (optional)
|
||||
|
||||
## When to Use Each Format
|
||||
|
||||
**Use format=index for:**
|
||||
- Quick overviews of observations by concept
|
||||
- Finding IDs for deeper investigation
|
||||
- Listing multiple results
|
||||
- **Token cost: ~50-100 per result**
|
||||
|
||||
**Use format=full for:**
|
||||
- Complete details including narrative, facts, files, concepts
|
||||
- Understanding the full context of specific observations
|
||||
- **Token cost: ~500-1000 per result**
|
||||
|
||||
## Example Response (format=index)
|
||||
|
||||
```json
|
||||
{
|
||||
"concept": "discovery",
|
||||
"count": 3,
|
||||
"format": "index",
|
||||
"results": [
|
||||
{
|
||||
"id": 1240,
|
||||
"type": "discovery",
|
||||
"title": "Worker service uses PM2 for process management",
|
||||
"subtitle": "Discovered persistent background worker pattern",
|
||||
"created_at_epoch": 1699564800000,
|
||||
"project": "claude-mem",
|
||||
"concepts": ["discovery", "how-it-works"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## How to Present Results
|
||||
|
||||
For format=index, present as a compact list:
|
||||
|
||||
```markdown
|
||||
Found 3 observations tagged with "discovery":
|
||||
|
||||
🔵 **#1240** Worker service uses PM2 for process management
|
||||
> Discovered persistent background worker pattern
|
||||
> Nov 9, 2024 • claude-mem
|
||||
> Tags: discovery, how-it-works
|
||||
|
||||
🔵 **#1241** FTS5 full-text search enables instant searches
|
||||
> SQLite FTS5 virtual tables provide sub-100ms search
|
||||
> Nov 9, 2024 • claude-mem
|
||||
> Tags: discovery, pattern
|
||||
```
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
|
||||
## Available Concepts
|
||||
|
||||
| Concept | Description | When to Use |
|
||||
|---------|-------------|-------------|
|
||||
| `discovery` | New discoveries and insights | Finding what was learned |
|
||||
| `problem-solution` | Problems and their solutions | Finding how issues were resolved |
|
||||
| `what-changed` | Change descriptions | Understanding what changed |
|
||||
| `how-it-works` | Explanations of mechanisms | Learning how things work |
|
||||
| `pattern` | Identified patterns | Finding design patterns |
|
||||
| `gotcha` | Edge cases and gotchas | Learning about pitfalls |
|
||||
| `change` | General changes | Tracking modifications |
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing concept parameter:**
|
||||
```json
|
||||
{"error": "Missing required parameter: concept"}
|
||||
```
|
||||
Fix: Add the concept parameter
|
||||
|
||||
**Invalid concept:**
|
||||
```json
|
||||
{"error": "Invalid concept: foobar. Valid concepts: discovery, problem-solution, what-changed, how-it-works, pattern, gotcha, change"}
|
||||
```
|
||||
Fix: Use one of the valid concept values
|
||||
|
||||
## Tips
|
||||
|
||||
1. Use format=index first to see overview
|
||||
2. Start with limit=5-10 to avoid token overload
|
||||
3. Combine concepts with type filtering for precision
|
||||
4. Use `discovery` for learning what was found during investigation
|
||||
5. Use `problem-solution` for finding how past issues were resolved
|
||||
|
||||
**Token Efficiency:**
|
||||
- Start with format=index (~50-100 tokens per result)
|
||||
- Use format=full only for relevant items (~500-1000 tokens per result)
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
@@ -0,0 +1,127 @@
|
||||
# Search by File
|
||||
|
||||
Find all work related to a specific file path.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What changes to auth/login.ts?"
|
||||
- User asks: "What work was done on this file?"
|
||||
- User asks: "Show me the history of src/services/worker.ts"
|
||||
- Looking for all observations that reference a file
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-file?filePath=src/services/worker-service.ts&format=index&limit=10"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- **filePath** (required): File path to search for (supports partial matching)
|
||||
- Full path: `src/services/worker-service.ts`
|
||||
- Partial path: `worker-service.ts`
|
||||
- Directory: `src/hooks/`
|
||||
- **format**: "index" (summary) or "full" (complete details). Default: "full"
|
||||
- **limit**: Number of results (default: 20, max: 100)
|
||||
- **project**: Filter by project name (optional)
|
||||
- **dateRange**: Filter by date range (optional)
|
||||
|
||||
## When to Use Each Format
|
||||
|
||||
**Use format=index for:**
|
||||
- Quick overviews of work on a file
|
||||
- Finding IDs for deeper investigation
|
||||
- Listing multiple changes
|
||||
- **Token cost: ~50-100 per result**
|
||||
|
||||
**Use format=full for:**
|
||||
- Complete details including narrative, facts, files, concepts
|
||||
- Understanding the full context of specific changes
|
||||
- **Token cost: ~500-1000 per result**
|
||||
|
||||
## Example Response (format=index)
|
||||
|
||||
```json
|
||||
{
|
||||
"filePath": "src/services/worker-service.ts",
|
||||
"count": 8,
|
||||
"format": "index",
|
||||
"results": [
|
||||
{
|
||||
"id": 1245,
|
||||
"type": "refactor",
|
||||
"title": "Simplified worker health check logic",
|
||||
"subtitle": "Removed redundant PM2 status check",
|
||||
"created_at_epoch": 1699564800000,
|
||||
"project": "claude-mem",
|
||||
"files": ["src/services/worker-service.ts", "src/services/worker-utils.ts"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## How to Present Results
|
||||
|
||||
For format=index, present as a compact list:
|
||||
|
||||
```markdown
|
||||
Found 8 observations related to "src/services/worker-service.ts":
|
||||
|
||||
🔄 **#1245** Simplified worker health check logic
|
||||
> Removed redundant PM2 status check
|
||||
> Nov 9, 2024 • claude-mem
|
||||
> Files: worker-service.ts, worker-utils.ts
|
||||
|
||||
🟣 **#1246** Added SSE endpoint for real-time updates
|
||||
> Implemented Server-Sent Events for viewer UI
|
||||
> Nov 8, 2024 • claude-mem
|
||||
> Files: worker-service.ts
|
||||
```
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
|
||||
## Partial Path Matching
|
||||
|
||||
The file path parameter supports partial matching:
|
||||
|
||||
```bash
|
||||
# These all match "src/services/worker-service.ts"
|
||||
curl -s "http://localhost:37777/api/search/by-file?filePath=worker-service.ts&format=index"
|
||||
curl -s "http://localhost:37777/api/search/by-file?filePath=services/worker&format=index"
|
||||
curl -s "http://localhost:37777/api/search/by-file?filePath=worker-service&format=index"
|
||||
```
|
||||
|
||||
## Directory Searches
|
||||
|
||||
Search for all work in a directory:
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-file?filePath=src/hooks/&format=index&limit=20"
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing filePath parameter:**
|
||||
```json
|
||||
{"error": "Missing required parameter: filePath"}
|
||||
```
|
||||
Fix: Add the filePath parameter
|
||||
|
||||
**No results found:**
|
||||
```json
|
||||
{"filePath": "nonexistent.ts", "count": 0, "results": []}
|
||||
```
|
||||
Response: "No observations found for 'nonexistent.ts'. Try a partial path or check the spelling."
|
||||
|
||||
## Tips
|
||||
|
||||
1. Use format=index first to see overview of all changes
|
||||
2. Start with partial paths (e.g., filename only) for broader matches
|
||||
3. Use full paths when you need specific file matches
|
||||
4. Combine with dateRange to see recent changes: `?filePath=worker.ts&dateRange[start]=2024-11-01`
|
||||
5. Use directory searches to see all work in a module
|
||||
|
||||
**Token Efficiency:**
|
||||
- Start with format=index (~50-100 tokens per result)
|
||||
- Use format=full only for relevant items (~500-1000 tokens per result)
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
@@ -0,0 +1,123 @@
|
||||
# Search by Type
|
||||
|
||||
Find observations by type: bugfix, feature, refactor, decision, discovery, or change.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What bugs did we fix?"
|
||||
- User asks: "What features did we add?"
|
||||
- User asks: "What decisions did we make?"
|
||||
- Looking for specific types of work
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=index&limit=5"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- **type** (required): One or more types (comma-separated)
|
||||
- `bugfix` - Bug fixes
|
||||
- `feature` - New features
|
||||
- `refactor` - Code refactoring
|
||||
- `decision` - Architectural/design decisions
|
||||
- `discovery` - Discoveries and insights
|
||||
- `change` - General changes
|
||||
- **format**: "index" (summary) or "full" (complete details). Default: "full"
|
||||
- **limit**: Number of results (default: 20, max: 100)
|
||||
- **project**: Filter by project name (optional)
|
||||
- **dateRange**: Filter by date range (optional)
|
||||
|
||||
## When to Use Each Format
|
||||
|
||||
**Use format=index for:**
|
||||
- Quick overviews of work by type
|
||||
- Finding IDs for deeper investigation
|
||||
- Listing multiple results
|
||||
- **Token cost: ~50-100 per result**
|
||||
|
||||
**Use format=full for:**
|
||||
- Complete details including narrative, facts, files, concepts
|
||||
- Understanding the full context of specific observations
|
||||
- **Token cost: ~500-1000 per result**
|
||||
|
||||
## Example Response (format=index)
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "bugfix",
|
||||
"count": 5,
|
||||
"format": "index",
|
||||
"results": [
|
||||
{
|
||||
"id": 1235,
|
||||
"type": "bugfix",
|
||||
"title": "Fixed token expiration edge case",
|
||||
"subtitle": "Handled race condition in refresh flow",
|
||||
"created_at_epoch": 1699564800000,
|
||||
"project": "api-server"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## How to Present Results
|
||||
|
||||
For format=index, present as a compact list with type emojis:
|
||||
|
||||
```markdown
|
||||
Found 5 bugfixes:
|
||||
|
||||
🔴 **#1235** Fixed token expiration edge case
|
||||
> Handled race condition in refresh flow
|
||||
> Nov 9, 2024 • api-server
|
||||
|
||||
🔴 **#1236** Resolved memory leak in worker
|
||||
> Fixed event listener cleanup
|
||||
> Nov 8, 2024 • worker-service
|
||||
```
|
||||
|
||||
**Type Emojis:**
|
||||
- 🔴 bugfix
|
||||
- 🟣 feature
|
||||
- 🔄 refactor
|
||||
- 🔵 discovery
|
||||
- 🧠 decision
|
||||
- ✅ change
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
|
||||
## Multiple Types
|
||||
|
||||
To search for multiple types:
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-type?type=bugfix,feature&format=index&limit=10"
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing type parameter:**
|
||||
```json
|
||||
{"error": "Missing required parameter: type"}
|
||||
```
|
||||
Fix: Add the type parameter
|
||||
|
||||
**Invalid type:**
|
||||
```json
|
||||
{"error": "Invalid type: foobar. Valid types: bugfix, feature, refactor, decision, discovery, change"}
|
||||
```
|
||||
Fix: Use one of the valid type values
|
||||
|
||||
## Tips
|
||||
|
||||
1. Use format=index first to see overview
|
||||
2. Start with limit=5-10 to avoid token overload
|
||||
3. Combine with dateRange for recent work: `?type=bugfix&dateRange[start]=2024-11-01`
|
||||
4. Use project filtering when working on one codebase
|
||||
|
||||
**Token Efficiency:**
|
||||
- Start with format=index (~50-100 tokens per result)
|
||||
- Use format=full only for relevant items (~500-1000 tokens per result)
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
@@ -0,0 +1,251 @@
|
||||
# Common Workflows
|
||||
|
||||
Step-by-step guides for typical user requests using the search API.
|
||||
|
||||
## Workflow 1: Understanding Past Work
|
||||
|
||||
**User asks:** "What did we do last session?" or "Catch me up on recent work"
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Get recent context** (fastest path):
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/context/recent?limit=3"
|
||||
```
|
||||
|
||||
2. **Present as narrative:**
|
||||
```markdown
|
||||
## Recent Work
|
||||
|
||||
### Session #545 - Nov 9, 2024
|
||||
Implemented JWT authentication system
|
||||
|
||||
**Completed:**
|
||||
- Added token-based auth with refresh tokens
|
||||
- Created JWT signing and verification logic
|
||||
|
||||
**Key Learning:** JWT expiration requires careful handling of refresh race conditions
|
||||
```
|
||||
|
||||
**Why this workflow:**
|
||||
- Single request gets both sessions and observations
|
||||
- Optimized for "catch me up" questions
|
||||
- ~1,500-2,500 tokens for 3 sessions
|
||||
|
||||
---
|
||||
|
||||
## Workflow 2: Finding Specific Bug Fixes
|
||||
|
||||
**User asks:** "What bugs did we fix?" or "Show me recent bug fixes"
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Search by type** (index format first):
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=index&limit=5"
|
||||
```
|
||||
|
||||
2. **Review index results**, identify relevant items
|
||||
|
||||
3. **Get full details** for specific bugs:
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=full&limit=1&offset=2"
|
||||
```
|
||||
|
||||
4. **Present findings:**
|
||||
```markdown
|
||||
Found 5 bug fixes:
|
||||
|
||||
🔴 **#1235** Fixed token expiration edge case
|
||||
> Handled race condition in refresh flow
|
||||
> Nov 9, 2024 • api-server
|
||||
|
||||
[Click for full details on #1235]
|
||||
```
|
||||
|
||||
**Why this workflow:**
|
||||
- Progressive disclosure: index first, full details selectively
|
||||
- Type-specific search is more efficient than generic search
|
||||
- ~250-500 tokens for index, ~750-1000 per full detail
|
||||
|
||||
---
|
||||
|
||||
## Workflow 3: Understanding File History
|
||||
|
||||
**User asks:** "What changes to auth/login.ts?" or "Show me work on this file"
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Search by file** (index format):
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-file?filePath=auth/login.ts&format=index&limit=10"
|
||||
```
|
||||
|
||||
2. **Review chronological changes**
|
||||
|
||||
3. **Get full details** for specific changes:
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/by-file?filePath=auth/login.ts&format=full&limit=1&offset=3"
|
||||
```
|
||||
|
||||
4. **Present as file timeline:**
|
||||
```markdown
|
||||
## History of auth/login.ts
|
||||
|
||||
🟣 **#1230** Added JWT authentication (Nov 9)
|
||||
🔴 **#1235** Fixed token expiration bug (Nov 9)
|
||||
🔄 **#1240** Refactored auth flow (Nov 8)
|
||||
```
|
||||
|
||||
**Why this workflow:**
|
||||
- File-specific search finds all related work
|
||||
- Index format shows chronological overview
|
||||
- Selective full details for deep dives
|
||||
|
||||
---
|
||||
|
||||
## Workflow 4: Timeline Investigation
|
||||
|
||||
**User asks:** "What was happening when we deployed?" or "Show me context around that bug fix"
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Find the event** using search:
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/observations?query=deployment&format=index&limit=5"
|
||||
```
|
||||
|
||||
2. **Note observation ID** (e.g., #1234)
|
||||
|
||||
3. **Get timeline context**:
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/timeline/context?anchor=1234&depth_before=10&depth_after=10"
|
||||
```
|
||||
|
||||
4. **Present as chronological narrative:**
|
||||
```markdown
|
||||
## Timeline: Deployment
|
||||
|
||||
### Before (10 records)
|
||||
**2:45 PM** - 🟣 Prepared deployment scripts
|
||||
**2:50 PM** - 💬 User asked: "Are we ready to deploy?"
|
||||
|
||||
### ⭐ Anchor Point (2:55 PM)
|
||||
🎯 **Observation #1234**: Deployed to production
|
||||
|
||||
### After (10 records)
|
||||
**3:00 PM** - 🔴 Fixed post-deployment routing issue
|
||||
```
|
||||
|
||||
**Why this workflow:**
|
||||
- Timeline shows temporal context (what happened before/after)
|
||||
- Captures causality between events
|
||||
- All record types (observations, sessions, prompts) interleaved
|
||||
|
||||
---
|
||||
|
||||
## Workflow 5: Quick Timeline (One Request)
|
||||
|
||||
**User asks:** "Timeline of authentication work"
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Use timeline-by-query** (auto mode):
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=auto&depth_before=10&depth_after=10"
|
||||
```
|
||||
|
||||
2. **Present timeline directly:**
|
||||
```markdown
|
||||
## Timeline: Authentication
|
||||
|
||||
**Best Match:** 🟣 Observation #1234 - Implemented JWT authentication
|
||||
|
||||
### Context (21 records)
|
||||
[... timeline around best match ...]
|
||||
```
|
||||
|
||||
**Why this workflow:**
|
||||
- Single request combines search + timeline
|
||||
- Fastest path when query is specific
|
||||
- Auto mode uses top result as anchor
|
||||
|
||||
**Alternative:** Use interactive mode for broad queries:
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/timeline/by-query?query=auth&mode=interactive&limit=5"
|
||||
```
|
||||
Then choose anchor manually.
|
||||
|
||||
---
|
||||
|
||||
## Workflow 6: Search Composition
|
||||
|
||||
**User asks:** "What features did we add to the authentication system recently?"
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Combine filters** for precision:
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/observations?query=authentication&type=feature&dateRange[start]=2024-11-01&format=index&limit=10"
|
||||
```
|
||||
|
||||
2. **Review filtered results**
|
||||
|
||||
3. **Get full details** for relevant features:
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/observations?query=authentication&type=feature&format=full&limit=1&offset=2"
|
||||
```
|
||||
|
||||
4. **Present findings:**
|
||||
```markdown
|
||||
Found 10 authentication features added in November:
|
||||
|
||||
🟣 **#1234** Implemented JWT authentication (Nov 9)
|
||||
🟣 **#1236** Added refresh token rotation (Nov 9)
|
||||
🟣 **#1238** Implemented OAuth2 flow (Nov 7)
|
||||
```
|
||||
|
||||
**Why this workflow:**
|
||||
- Multiple filters narrow results before requesting full details
|
||||
- Type + query + dateRange = precise targeting
|
||||
- Progressive disclosure: index first, full details selectively
|
||||
|
||||
---
|
||||
|
||||
## Workflow Selection Guide
|
||||
|
||||
| User Request | Workflow | Operation | Token Cost |
|
||||
|--------------|----------|-----------|------------|
|
||||
| "What did we do last session?" | #1 | recent-context | 1,500-2,500 |
|
||||
| "What bugs did we fix?" | #2 | by-type | 500-3,000 |
|
||||
| "What changes to file.ts?" | #3 | by-file | 500-3,000 |
|
||||
| "What was happening then?" | #4 | search → timeline | 3,500-6,000 |
|
||||
| "Timeline of X work" | #5 | timeline-by-query | 3,000-4,000 |
|
||||
| "Recent features added?" | #6 | observations + filters | 500-3,000 |
|
||||
|
||||
## General Principles
|
||||
|
||||
1. **Start with index format** - Always use `format=index` first
|
||||
2. **Use specialized tools** - by-type, by-file, by-concept when applicable
|
||||
3. **Compose operations** - Combine search + timeline for investigations
|
||||
4. **Filter early** - Use type, dateRange, project to narrow before expanding
|
||||
5. **Progressive disclosure** - Load full details only for relevant items
|
||||
|
||||
## Token Budget Awareness
|
||||
|
||||
**Quick queries** (500-1,500 tokens):
|
||||
- Recent context (limit=3)
|
||||
- Index search (limit=5-10)
|
||||
- Filtered searches
|
||||
|
||||
**Medium queries** (1,500-4,000 tokens):
|
||||
- Recent context (limit=5-10)
|
||||
- Full details (3-5 items)
|
||||
- Timeline (depth 10/10)
|
||||
|
||||
**Deep queries** (4,000-8,000 tokens):
|
||||
- Timeline (depth 20/20)
|
||||
- Full details (10+ items)
|
||||
- Multiple composed operations
|
||||
|
||||
Always start with minimal token investment, expand only when needed.
|
||||
@@ -0,0 +1,403 @@
|
||||
# Response Formatting Guidelines
|
||||
|
||||
How to present search results to users for maximum clarity and usefulness.
|
||||
|
||||
## General Principles
|
||||
|
||||
1. **Progressive disclosure** - Show index results first, full details on demand
|
||||
2. **Visual hierarchy** - Use emojis, bold, and structure for scannability
|
||||
3. **Context-aware** - Tailor presentation to user's question
|
||||
4. **Actionable** - Include IDs for follow-up queries
|
||||
5. **Token-efficient** - Balance detail with token budget
|
||||
|
||||
---
|
||||
|
||||
## Format: Index Results
|
||||
|
||||
**When to use:** First response to searches, overviews, multiple results
|
||||
|
||||
**Structure:**
|
||||
```markdown
|
||||
Found {count} results for "{query}":
|
||||
|
||||
{emoji} **#{id}** {title}
|
||||
> {subtitle}
|
||||
> {date} • {project}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```markdown
|
||||
Found 5 results for "authentication":
|
||||
|
||||
🟣 **#1234** Implemented JWT authentication
|
||||
> Added token-based auth with refresh tokens
|
||||
> Nov 9, 2024 • api-server
|
||||
|
||||
🔴 **#1235** Fixed token expiration edge case
|
||||
> Handled race condition in refresh flow
|
||||
> Nov 9, 2024 • api-server
|
||||
```
|
||||
|
||||
**Type Emojis:**
|
||||
- 🔴 bugfix
|
||||
- 🟣 feature
|
||||
- 🔄 refactor
|
||||
- 🔵 discovery
|
||||
- 🧠 decision
|
||||
- ✅ change
|
||||
- 🎯 session
|
||||
- 💬 prompt
|
||||
|
||||
**What to include:**
|
||||
- ✅ ID (for follow-up)
|
||||
- ✅ Type emoji
|
||||
- ✅ Title
|
||||
- ✅ Subtitle (if available)
|
||||
- ✅ Date (human-readable)
|
||||
- ✅ Project name
|
||||
- ❌ Don't include full narrative/facts/files in index format
|
||||
|
||||
---
|
||||
|
||||
## Format: Full Results
|
||||
|
||||
**When to use:** User requests details, specific items selected from index
|
||||
|
||||
**Structure:**
|
||||
```markdown
|
||||
## {emoji} {type} #{id}: {title}
|
||||
|
||||
**Summary:** {subtitle}
|
||||
|
||||
**What happened:**
|
||||
{narrative}
|
||||
|
||||
**Key Facts:**
|
||||
- {fact1}
|
||||
- {fact2}
|
||||
|
||||
**Files modified:**
|
||||
- {file1}
|
||||
- {file2}
|
||||
|
||||
**Concepts:** {concepts}
|
||||
|
||||
**Date:** {human_readable_date}
|
||||
**Project:** {project}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```markdown
|
||||
## 🟣 Feature #1234: Implemented JWT authentication
|
||||
|
||||
**Summary:** Added token-based auth with refresh tokens
|
||||
|
||||
**What happened:**
|
||||
Implemented a complete JWT authentication system with access and refresh tokens. Access tokens expire after 15 minutes, refresh tokens after 7 days. Added token signing with RS256 algorithm and proper key rotation infrastructure.
|
||||
|
||||
**Key Facts:**
|
||||
- Access tokens use 15-minute expiration
|
||||
- Refresh tokens stored in httpOnly cookies
|
||||
- RS256 algorithm with key rotation support
|
||||
- Token refresh endpoint handles race conditions gracefully
|
||||
|
||||
**Files modified:**
|
||||
- src/auth/jwt.ts (created)
|
||||
- src/auth/refresh.ts (created)
|
||||
- src/middleware/auth.ts (modified)
|
||||
|
||||
**Concepts:** how-it-works, pattern
|
||||
|
||||
**Date:** November 9, 2024 at 2:55 PM
|
||||
**Project:** api-server
|
||||
```
|
||||
|
||||
**What to include:**
|
||||
- ✅ Full title with emoji and ID
|
||||
- ✅ Summary/subtitle
|
||||
- ✅ Complete narrative
|
||||
- ✅ All key facts
|
||||
- ✅ All files (with status: created/modified/deleted)
|
||||
- ✅ Concepts/tags
|
||||
- ✅ Precise timestamp
|
||||
- ✅ Project name
|
||||
|
||||
---
|
||||
|
||||
## Format: Timeline Results
|
||||
|
||||
**When to use:** Temporal investigations, "what was happening" questions
|
||||
|
||||
**Structure:**
|
||||
```markdown
|
||||
## Timeline: {anchor_description}
|
||||
|
||||
### Before ({count} records)
|
||||
|
||||
**{time}** - {emoji} {type} #{id}: {title}
|
||||
**{time}** - {emoji} {type} #{id}: {title}
|
||||
|
||||
### ⭐ Anchor Point ({time})
|
||||
{emoji} **{type} #{id}**: {title}
|
||||
|
||||
### After ({count} records)
|
||||
|
||||
**{time}** - {emoji} {type} #{id}: {title}
|
||||
**{time}** - {emoji} {type} #{id}: {title}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```markdown
|
||||
## Timeline: Deployment
|
||||
|
||||
### Before (10 records)
|
||||
|
||||
**2:30 PM** - 🟣 #1230: Prepared deployment scripts
|
||||
**2:45 PM** - 🔄 #1232: Updated configuration files
|
||||
**2:50 PM** - 💬 User asked: "Are we ready to deploy?"
|
||||
|
||||
### ⭐ Anchor Point (2:55 PM)
|
||||
🎯 **Session #545**: Deployed to production
|
||||
|
||||
### After (10 records)
|
||||
|
||||
**3:00 PM** - 🔴 #1235: Fixed post-deployment routing issue
|
||||
**3:10 PM** - 🔵 #1236: Discovered caching behavior in production
|
||||
**3:15 PM** - 🧠 #1237: Decided to add health check endpoint
|
||||
```
|
||||
|
||||
**What to include:**
|
||||
- ✅ Chronological ordering (oldest to newest)
|
||||
- ✅ Human-readable times (not epochs)
|
||||
- ✅ Clear anchor point marker (⭐)
|
||||
- ✅ Mix of all record types (observations, sessions, prompts)
|
||||
- ✅ Concise titles (not full narratives)
|
||||
- ✅ Type emojis for quick scanning
|
||||
|
||||
---
|
||||
|
||||
## Format: Session Summaries
|
||||
|
||||
**When to use:** Recent context, "what did we do" questions
|
||||
|
||||
**Structure:**
|
||||
```markdown
|
||||
## Recent Work on {project}
|
||||
|
||||
### 🎯 Session #{id} - {date}
|
||||
|
||||
**Request:** {user_request}
|
||||
|
||||
**Completed:**
|
||||
- {completion1}
|
||||
- {completion2}
|
||||
|
||||
**Key Learning:** {learning}
|
||||
|
||||
**Observations:**
|
||||
- {emoji} **#{obs_id}** {obs_title}
|
||||
- Files: {file_list}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```markdown
|
||||
## Recent Work on api-server
|
||||
|
||||
### 🎯 Session #545 - November 9, 2024
|
||||
|
||||
**Request:** Add JWT authentication with refresh tokens
|
||||
|
||||
**Completed:**
|
||||
- Implemented token-based auth with refresh logic
|
||||
- Added JWT signing and verification
|
||||
- Created refresh token rotation
|
||||
|
||||
**Key Learning:** JWT expiration requires careful handling of refresh race conditions
|
||||
|
||||
**Observations:**
|
||||
- 🟣 **#1234** Implemented JWT authentication
|
||||
- Files: jwt.ts, refresh.ts, middleware/auth.ts
|
||||
- 🔴 **#1235** Fixed token expiration edge case
|
||||
- Files: refresh.ts
|
||||
```
|
||||
|
||||
**What to include:**
|
||||
- ✅ Session ID and date
|
||||
- ✅ Original user request
|
||||
- ✅ What was completed (bulleted list)
|
||||
- ✅ Key learnings/insights
|
||||
- ✅ Linked observations with file lists
|
||||
- ✅ Clear hierarchy (session → observations)
|
||||
|
||||
---
|
||||
|
||||
## Format: User Prompts
|
||||
|
||||
**When to use:** "What did I ask" questions, prompt searches
|
||||
|
||||
**Structure:**
|
||||
```markdown
|
||||
Found {count} user prompts:
|
||||
|
||||
💬 **Prompt #{id}** (Session #{session_id})
|
||||
> "{preview_text}"
|
||||
> {date} • {project}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```markdown
|
||||
Found 5 user prompts about "authentication":
|
||||
|
||||
💬 **Prompt #1250** (Session #545)
|
||||
> "How do I implement JWT authentication with refresh tokens? I need to handle token expiration..."
|
||||
> Nov 9, 2024 • api-server
|
||||
|
||||
💬 **Prompt #1251** (Session #546)
|
||||
> "The auth tokens are expiring too quickly. Can you help debug the refresh flow?"
|
||||
> Nov 8, 2024 • api-server
|
||||
```
|
||||
|
||||
**What to include:**
|
||||
- ✅ Prompt ID
|
||||
- ✅ Session ID (for context linking)
|
||||
- ✅ Preview text (200 chars for index, full text for full format)
|
||||
- ✅ Date and project
|
||||
- ✅ Quote formatting for prompt text
|
||||
|
||||
---
|
||||
|
||||
## Error Responses
|
||||
|
||||
**No results found:**
|
||||
```markdown
|
||||
No results found for "{query}". Try:
|
||||
- Different search terms
|
||||
- Broader keywords
|
||||
- Checking spelling
|
||||
- Using partial paths (for file searches)
|
||||
```
|
||||
|
||||
**Service unavailable:**
|
||||
```markdown
|
||||
The search service isn't available. Check if the worker is running:
|
||||
|
||||
```bash
|
||||
pm2 list
|
||||
```
|
||||
|
||||
If the worker is stopped, restart it:
|
||||
|
||||
```bash
|
||||
npm run worker:restart
|
||||
```
|
||||
```
|
||||
|
||||
**Invalid parameters:**
|
||||
```markdown
|
||||
Invalid search parameters:
|
||||
- {parameter}: {error_message}
|
||||
|
||||
See the [API help](help.md) for valid parameter options.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Context-Aware Presentation
|
||||
|
||||
Tailor formatting to user's question:
|
||||
|
||||
**"What bugs did we fix?"**
|
||||
→ Use index format, emphasize date/type, group by recency
|
||||
|
||||
**"How did we implement X?"**
|
||||
→ Use full format for best match, include complete narrative and files
|
||||
|
||||
**"What was happening when..."**
|
||||
→ Use timeline format, emphasize chronology and causality
|
||||
|
||||
**"Catch me up on recent work"**
|
||||
→ Use session summary format, focus on high-level accomplishments
|
||||
|
||||
---
|
||||
|
||||
## Token Budget Guidelines
|
||||
|
||||
**Minimal presentation (~100-200 tokens):**
|
||||
- Index format with 3-5 results
|
||||
- Compact list structure
|
||||
- Essential metadata only
|
||||
|
||||
**Standard presentation (~500-1,000 tokens):**
|
||||
- Index format with 10-15 results
|
||||
- Include subtitles and context
|
||||
- Clear formatting and emojis
|
||||
|
||||
**Detailed presentation (~1,500-3,000 tokens):**
|
||||
- Full format for 2-3 items
|
||||
- Complete narratives and facts
|
||||
- Timeline with 20-30 records
|
||||
|
||||
**Comprehensive presentation (~5,000+ tokens):**
|
||||
- Multiple full results
|
||||
- Deep timelines (40+ records)
|
||||
- Session summaries with observations
|
||||
|
||||
Always start minimal, expand only when needed.
|
||||
|
||||
---
|
||||
|
||||
## Markdown Best Practices
|
||||
|
||||
1. **Use headers (##, ###)** for hierarchy
|
||||
2. **Bold important elements** (IDs, titles, dates)
|
||||
3. **Quote user text** (prompts, questions)
|
||||
4. **Bullet lists** for facts and files
|
||||
5. **Code blocks** for commands and examples
|
||||
6. **Emojis** for type indicators
|
||||
7. **Horizontal rules (---)** for section breaks
|
||||
8. **Blockquotes (>)** for subtitles and previews
|
||||
|
||||
---
|
||||
|
||||
## Examples by Use Case
|
||||
|
||||
### Use Case 1: Quick Overview
|
||||
User: "What did we do last session?"
|
||||
|
||||
```markdown
|
||||
## Recent Work
|
||||
|
||||
### 🎯 Session #545 - November 9, 2024
|
||||
Implemented JWT authentication system
|
||||
|
||||
**Key accomplishment:** Added token-based auth with refresh tokens
|
||||
**Key learning:** JWT expiration requires careful handling of refresh race conditions
|
||||
```
|
||||
|
||||
### Use Case 2: Specific Investigation
|
||||
User: "How did we implement JWT authentication?"
|
||||
|
||||
```markdown
|
||||
## 🟣 Feature #1234: Implemented JWT authentication
|
||||
|
||||
**What happened:**
|
||||
Implemented a complete JWT authentication system with access and refresh tokens. Access tokens expire after 15 minutes, refresh tokens after 7 days. Added token signing with RS256 algorithm.
|
||||
|
||||
**Files:**
|
||||
- src/auth/jwt.ts (created)
|
||||
- src/auth/refresh.ts (created)
|
||||
- src/middleware/auth.ts (modified)
|
||||
|
||||
**Key insight:** Refresh race conditions require atomic token exchange logic.
|
||||
```
|
||||
|
||||
### Use Case 3: Timeline Investigation
|
||||
User: "What was happening around the deployment?"
|
||||
|
||||
```markdown
|
||||
## Timeline: Deployment
|
||||
|
||||
[... chronological timeline with before/after context ...]
|
||||
```
|
||||
|
||||
Choose presentation style based on user's question and information needs.
|
||||
@@ -0,0 +1,175 @@
|
||||
# API Help
|
||||
|
||||
Get comprehensive API documentation for all search endpoints.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What search operations are available?"
|
||||
- User asks: "How do I use the search API?"
|
||||
- Need reference documentation for endpoints
|
||||
- Want to see all available parameters
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/help"
|
||||
```
|
||||
|
||||
## Response Structure
|
||||
|
||||
Returns complete API documentation:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "5.4.0",
|
||||
"base_url": "http://localhost:37777/api",
|
||||
"endpoints": [
|
||||
{
|
||||
"path": "/search/observations",
|
||||
"method": "GET",
|
||||
"description": "Search observations using full-text search",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "query",
|
||||
"required": true,
|
||||
"type": "string",
|
||||
"description": "Search terms"
|
||||
},
|
||||
{
|
||||
"name": "format",
|
||||
"required": false,
|
||||
"type": "string",
|
||||
"default": "full",
|
||||
"options": ["index", "full"],
|
||||
"description": "Response format"
|
||||
}
|
||||
],
|
||||
"example": "curl -s \"http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5\""
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## How to Present Results
|
||||
|
||||
Present as reference documentation:
|
||||
|
||||
```markdown
|
||||
## claude-mem Search API Reference (v5.4.0)
|
||||
|
||||
Base URL: `http://localhost:37777/api`
|
||||
|
||||
### Search Operations
|
||||
|
||||
**1. Search Observations**
|
||||
- **Endpoint:** `GET /search/observations`
|
||||
- **Description:** Search observations using full-text search
|
||||
- **Parameters:**
|
||||
- `query` (required, string): Search terms
|
||||
- `format` (optional, string): "index" or "full" (default: "full")
|
||||
- `limit` (optional, number): Max results (default: 20, max: 100)
|
||||
- **Example:**
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
|
||||
```
|
||||
|
||||
[... continue for all endpoints ...]
|
||||
```
|
||||
|
||||
## Endpoint Categories
|
||||
|
||||
The API help response organizes endpoints by category:
|
||||
|
||||
1. **Full-Text Search**
|
||||
- `/search/observations`
|
||||
- `/search/sessions`
|
||||
- `/search/prompts`
|
||||
|
||||
2. **Filtered Search**
|
||||
- `/search/by-type`
|
||||
- `/search/by-concept`
|
||||
- `/search/by-file`
|
||||
|
||||
3. **Context Retrieval**
|
||||
- `/context/recent`
|
||||
- `/timeline/context`
|
||||
- `/timeline/by-query`
|
||||
|
||||
4. **Utilities**
|
||||
- `/help`
|
||||
|
||||
## Common Parameters
|
||||
|
||||
Many endpoints share these parameters:
|
||||
|
||||
- **format**: "index" (summary) or "full" (complete details)
|
||||
- **limit**: Number of results to return
|
||||
- **offset**: Number of results to skip (for pagination)
|
||||
- **project**: Filter by project name
|
||||
- **dateRange**: Filter by date range
|
||||
- `dateRange[start]`: Start date (ISO string or epoch)
|
||||
- `dateRange[end]`: End date (ISO string or epoch)
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Worker not running:**
|
||||
Connection refused error. Response: "The search API isn't available. Check if worker is running: `pm2 list`"
|
||||
|
||||
**Invalid endpoint:**
|
||||
```json
|
||||
{"error": "Not found"}
|
||||
```
|
||||
Response: "Invalid API endpoint. Use /api/help to see available endpoints."
|
||||
|
||||
## Tips
|
||||
|
||||
1. Save help response for reference during investigation
|
||||
2. Use examples as starting point for your queries
|
||||
3. Check required parameters before making requests
|
||||
4. Refer to format options for each endpoint
|
||||
5. All endpoints use GET method with query parameters
|
||||
|
||||
**Token Efficiency:**
|
||||
- Help response: ~2,000-3,000 tokens (complete API reference)
|
||||
- Use sparingly - refer to operation-specific docs instead
|
||||
- Keep help response cached for repeated reference
|
||||
|
||||
## When to Use Help
|
||||
|
||||
**Use help when:**
|
||||
- Starting to use the search API
|
||||
- Need complete parameter reference
|
||||
- Forgot which endpoints are available
|
||||
- Want to see all options at once
|
||||
|
||||
**Don't use help when:**
|
||||
- You know which operation you need (use operation-specific docs)
|
||||
- Just need examples (use common-workflows.md)
|
||||
- Token budget is limited (help is comprehensive)
|
||||
|
||||
## Alternative to Help Endpoint
|
||||
|
||||
Instead of calling `/api/help`, you can:
|
||||
|
||||
1. **Use SKILL.md** - Quick decision guide with operation links
|
||||
2. **Use operation docs** - Detailed guides for specific endpoints
|
||||
3. **Use common-workflows.md** - Step-by-step examples
|
||||
4. **Use formatting.md** - Response presentation templates
|
||||
|
||||
The help endpoint is most useful when you need complete API reference in one response.
|
||||
|
||||
## API Versioning
|
||||
|
||||
The help response includes version information:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "5.4.0",
|
||||
"skill_migration": true,
|
||||
"deprecated": {
|
||||
"mcp_tools": "Replaced by HTTP API in v5.4.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Check version to ensure compatibility with documentation.
|
||||
@@ -12,7 +12,7 @@ Search all observations using natural language queries.
|
||||
## Command
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=20"
|
||||
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
@@ -28,10 +28,12 @@ curl -s "http://localhost:37777/api/search/observations?query=authentication&for
|
||||
- Quick overviews
|
||||
- Finding IDs for deeper investigation
|
||||
- Listing multiple results
|
||||
- **Token cost: ~50-100 per result**
|
||||
|
||||
**Use format=full for:**
|
||||
- Complete details including narrative, facts, files, concepts
|
||||
- Understanding the full context of specific observations
|
||||
- **Token cost: ~500-1000 per result**
|
||||
|
||||
## Example Response (format=index)
|
||||
|
||||
@@ -72,7 +74,7 @@ Found 5 results for "authentication":
|
||||
|
||||
**Include:** ID (for follow-up), type emoji (🔴 bugfix, 🟣 feature, 🔄 refactor, 🔵 discovery, 🧠 decision, ✅ change), title, subtitle, date, project.
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
For complete formatting guidelines, see formatting.md (documentation coming soon).
|
||||
|
||||
## Error Handling
|
||||
|
||||
@@ -94,3 +96,8 @@ Response: "No results found for 'foobar'. Try different search terms."
|
||||
2. Start with format=index and limit=5-10
|
||||
3. Use project filtering when working on one codebase
|
||||
4. If no results, try broader terms or check spelling
|
||||
|
||||
**Token Efficiency:**
|
||||
- Start with format=index (~50-100 tokens per result)
|
||||
- Use format=full only for relevant items (~500-1000 tokens per result)
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
@@ -0,0 +1,125 @@
|
||||
# Search User Prompts (Full-Text)
|
||||
|
||||
Search raw user prompts to find what was actually asked across all sessions.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What did I ask about authentication?"
|
||||
- User asks: "Find my question about database migrations"
|
||||
- User asks: "When did I ask about testing?"
|
||||
- Looking for specific user questions or requests
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/prompts?query=authentication&format=index&limit=5"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- **query** (required): Search terms (e.g., "authentication", "how do I", "bug fix")
|
||||
- **format**: "index" (truncated prompts) or "full" (complete prompt text). Default: "full"
|
||||
- **limit**: Number of results (default: 20, max: 100)
|
||||
- **project**: Filter by project name (optional)
|
||||
- **dateRange**: Filter by date range (optional)
|
||||
|
||||
## When to Use Each Format
|
||||
|
||||
**Use format=index for:**
|
||||
- Quick overviews of what was asked
|
||||
- Finding prompt IDs for full text
|
||||
- Listing multiple prompts
|
||||
- **Token cost: ~50-100 per result (truncated to 200 chars)**
|
||||
|
||||
**Use format=full for:**
|
||||
- Complete prompt text
|
||||
- Understanding the full user request
|
||||
- **Token cost: Variable (depends on prompt length, typically 100-300 tokens)**
|
||||
|
||||
## Example Response (format=index)
|
||||
|
||||
```json
|
||||
{
|
||||
"query": "authentication",
|
||||
"count": 5,
|
||||
"format": "index",
|
||||
"results": [
|
||||
{
|
||||
"id": 1250,
|
||||
"session_id": "S545",
|
||||
"prompt_preview": "How do I implement JWT authentication with refresh tokens? I need to handle token expiration...",
|
||||
"created_at_epoch": 1699564800000,
|
||||
"project": "api-server"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## How to Present Results
|
||||
|
||||
For format=index, present as a compact list:
|
||||
|
||||
```markdown
|
||||
Found 5 user prompts about "authentication":
|
||||
|
||||
💬 **Prompt #1250** (Session #545)
|
||||
> "How do I implement JWT authentication with refresh tokens? I need to handle token expiration..."
|
||||
> Nov 9, 2024 • api-server
|
||||
|
||||
💬 **Prompt #1251** (Session #546)
|
||||
> "The auth tokens are expiring too quickly. Can you help debug the refresh flow?"
|
||||
> Nov 8, 2024 • api-server
|
||||
```
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
|
||||
## What Gets Searched
|
||||
|
||||
User prompts search covers:
|
||||
- All user messages sent to Claude Code
|
||||
- Raw text as typed by the user
|
||||
- Multi-turn conversations (each message is a separate prompt)
|
||||
- Questions, requests, commands, and clarifications
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing query parameter:**
|
||||
```json
|
||||
{"error": "Missing required parameter: query"}
|
||||
```
|
||||
Fix: Add the query parameter
|
||||
|
||||
**No results found:**
|
||||
```json
|
||||
{"query": "foobar", "count": 0, "results": []}
|
||||
```
|
||||
Response: "No user prompts found for 'foobar'. Try different search terms."
|
||||
|
||||
## Tips
|
||||
|
||||
1. Use exact phrases in quotes: `?query="how do I"` for precise matches
|
||||
2. Start with format=index to see preview, then get full text if needed
|
||||
3. Use dateRange to find recent questions: `?query=bug&dateRange[start]=2024-11-01`
|
||||
4. Prompts show what was asked, sessions/observations show what was done
|
||||
5. Combine with session search to see both question and answer
|
||||
|
||||
**Token Efficiency:**
|
||||
- Start with format=index (~50-100 tokens per result, prompt truncated to 200 chars)
|
||||
- Use format=full only for relevant items (100-300 tokens per result)
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
|
||||
## When to Use Prompts vs Sessions
|
||||
|
||||
**Use prompts search when:**
|
||||
- Looking for specific user questions
|
||||
- Trying to remember what was asked
|
||||
- Finding original request wording
|
||||
|
||||
**Use sessions search when:**
|
||||
- Looking for what was accomplished
|
||||
- Understanding work summaries
|
||||
- Getting high-level context
|
||||
|
||||
**Combine both when:**
|
||||
- Understanding the full conversation (what was asked + what was done)
|
||||
- Investigating how a request was interpreted
|
||||
@@ -0,0 +1,134 @@
|
||||
# Get Recent Context
|
||||
|
||||
Get recent session summaries and observations for a project.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What did we do last session?"
|
||||
- User asks: "What have we been working on recently?"
|
||||
- User asks: "Catch me up on recent work"
|
||||
- Starting a new session and need context
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/context/recent?project=api-server&limit=3"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- **project**: Project name (defaults to current working directory basename)
|
||||
- **limit**: Number of recent sessions to retrieve (default: 3, max: 10)
|
||||
|
||||
## Response Structure
|
||||
|
||||
Returns combined context from recent sessions:
|
||||
|
||||
```json
|
||||
{
|
||||
"project": "api-server",
|
||||
"limit": 3,
|
||||
"sessions": [
|
||||
{
|
||||
"id": 545,
|
||||
"session_id": "S545",
|
||||
"title": "Implemented JWT authentication system",
|
||||
"request": "Add JWT authentication with refresh tokens",
|
||||
"completion": "Implemented token-based auth with refresh logic",
|
||||
"learnings": "JWT expiration requires careful handling of refresh race conditions",
|
||||
"created_at_epoch": 1699564800000,
|
||||
"observations": [
|
||||
{
|
||||
"id": 1234,
|
||||
"type": "feature",
|
||||
"title": "Implemented JWT authentication",
|
||||
"subtitle": "Added token-based auth with refresh tokens",
|
||||
"files": ["src/auth/jwt.ts", "src/auth/refresh.ts"]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## How to Present Results
|
||||
|
||||
Present as a chronological narrative:
|
||||
|
||||
```markdown
|
||||
## Recent Work on api-server
|
||||
|
||||
### Session #545 - Nov 9, 2024
|
||||
**Request:** Add JWT authentication with refresh tokens
|
||||
|
||||
**Completed:**
|
||||
- Implemented token-based auth with refresh logic
|
||||
- Added JWT signing and verification
|
||||
- Created refresh token rotation
|
||||
|
||||
**Key Learning:** JWT expiration requires careful handling of refresh race conditions
|
||||
|
||||
**Observations:**
|
||||
- 🟣 **#1234** Implemented JWT authentication
|
||||
- Files: jwt.ts, refresh.ts
|
||||
```
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
|
||||
## Default Project Detection
|
||||
|
||||
If no project parameter is provided, uses current working directory:
|
||||
|
||||
```bash
|
||||
# Auto-detects project from current directory
|
||||
curl -s "http://localhost:37777/api/context/recent?limit=3"
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
**No sessions found:**
|
||||
```json
|
||||
{"project": "new-project", "sessions": []}
|
||||
```
|
||||
Response: "No recent sessions found for 'new-project'. This might be a new project."
|
||||
|
||||
**Worker not running:**
|
||||
Connection refused error. Inform user to check if worker is running: `pm2 list`
|
||||
|
||||
## Tips
|
||||
|
||||
1. Start with limit=3 for quick overview (default)
|
||||
2. Increase to limit=5-10 for deeper context
|
||||
3. Recent context is perfect for session start
|
||||
4. Combines both sessions and observations in one request
|
||||
5. Use this when user asks "what did we do last time?"
|
||||
|
||||
**Token Efficiency:**
|
||||
- limit=3 sessions: ~1,500-2,500 tokens (includes observations)
|
||||
- limit=5 sessions: ~2,500-4,000 tokens
|
||||
- limit=10 sessions: ~5,000-8,000 tokens
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
|
||||
## When to Use Recent Context
|
||||
|
||||
**Use recent-context when:**
|
||||
- Starting a new session
|
||||
- User asks about recent work
|
||||
- Need quick catch-up on project activity
|
||||
- Want both sessions and observations together
|
||||
|
||||
**Don't use recent-context when:**
|
||||
- Looking for specific topics (use search instead)
|
||||
- Need timeline around specific event (use timeline instead)
|
||||
- Want only observations or only sessions (use search operations)
|
||||
|
||||
## Comparison with Other Operations
|
||||
|
||||
| Operation | Use Case | Token Cost |
|
||||
|-----------|----------|------------|
|
||||
| recent-context | Quick catch-up on recent work | 1,500-4,000 |
|
||||
| sessions search | Find sessions by topic | 50-100 per result (index) |
|
||||
| observations search | Find specific implementations | 50-100 per result (index) |
|
||||
| timeline | Context around specific point | 3,000-6,000 |
|
||||
|
||||
Recent context is optimized for "what happened recently?" questions with minimal token usage.
|
||||
@@ -0,0 +1,124 @@
|
||||
# Search Sessions (Full-Text)
|
||||
|
||||
Search session summaries using natural language queries.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What did we work on last week?"
|
||||
- User asks: "What sessions involved database work?"
|
||||
- User asks: "Show me sessions where we fixed bugs"
|
||||
- Looking for past sessions by topic or theme
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/search/sessions?query=authentication&format=index&limit=5"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- **query** (required): Search terms (e.g., "authentication", "database migration", "bug fixes")
|
||||
- **format**: "index" (summary) or "full" (complete details). Default: "full"
|
||||
- **limit**: Number of results (default: 20, max: 100)
|
||||
- **project**: Filter by project name (optional)
|
||||
- **dateRange**: Filter by date range (optional)
|
||||
|
||||
## When to Use Each Format
|
||||
|
||||
**Use format=index for:**
|
||||
- Quick overviews of past sessions
|
||||
- Finding session IDs for deeper investigation
|
||||
- Listing multiple sessions
|
||||
- **Token cost: ~50-100 per result**
|
||||
|
||||
**Use format=full for:**
|
||||
- Complete session summaries with requests, completions, learnings
|
||||
- Understanding the full context of a session
|
||||
- **Token cost: ~500-1000 per result**
|
||||
|
||||
## Example Response (format=index)
|
||||
|
||||
```json
|
||||
{
|
||||
"query": "authentication",
|
||||
"count": 3,
|
||||
"format": "index",
|
||||
"results": [
|
||||
{
|
||||
"id": 545,
|
||||
"session_id": "S545",
|
||||
"title": "Implemented JWT authentication system",
|
||||
"subtitle": "Added token-based auth with refresh tokens",
|
||||
"created_at_epoch": 1699564800000,
|
||||
"project": "api-server"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## How to Present Results
|
||||
|
||||
For format=index, present as a compact list:
|
||||
|
||||
```markdown
|
||||
Found 3 sessions about "authentication":
|
||||
|
||||
🎯 **Session #545** Implemented JWT authentication system
|
||||
> Added token-based auth with refresh tokens
|
||||
> Nov 9, 2024 • api-server
|
||||
|
||||
🎯 **Session #546** Fixed authentication token expiration
|
||||
> Resolved race condition in token refresh flow
|
||||
> Nov 8, 2024 • api-server
|
||||
```
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
|
||||
## Session Summary Structure
|
||||
|
||||
Full session summaries include:
|
||||
|
||||
- **Session request**: What the user asked for
|
||||
- **What was completed**: Summary of work done
|
||||
- **Key learnings**: Important insights and discoveries
|
||||
- **Files modified**: List of changed files
|
||||
- **Observations**: Links to detailed observations
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing query parameter:**
|
||||
```json
|
||||
{"error": "Missing required parameter: query"}
|
||||
```
|
||||
Fix: Add the query parameter
|
||||
|
||||
**No results found:**
|
||||
```json
|
||||
{"query": "foobar", "count": 0, "results": []}
|
||||
```
|
||||
Response: "No sessions found for 'foobar'. Try different search terms."
|
||||
|
||||
## Tips
|
||||
|
||||
1. Be specific: "JWT authentication implementation" > "auth"
|
||||
2. Start with format=index and limit=5-10
|
||||
3. Use dateRange for recent sessions: `?query=auth&dateRange[start]=2024-11-01`
|
||||
4. Sessions provide high-level overview, observations provide details
|
||||
5. Use project filtering when working on one codebase
|
||||
|
||||
**Token Efficiency:**
|
||||
- Start with format=index (~50-100 tokens per result)
|
||||
- Use format=full only for relevant items (~500-1000 tokens per result)
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
|
||||
## When to Use Sessions vs Observations
|
||||
|
||||
**Use sessions search when:**
|
||||
- Looking for high-level work summaries
|
||||
- Understanding what was done in past sessions
|
||||
- Getting overview of recent activity
|
||||
|
||||
**Use observations search when:**
|
||||
- Looking for specific implementation details
|
||||
- Finding bugs, features, or decisions
|
||||
- Need fine-grained context about code changes
|
||||
@@ -0,0 +1,192 @@
|
||||
# Timeline by Query
|
||||
|
||||
Search for observations and get timeline context in a single request. Combines search + timeline into one operation.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What was happening when we worked on authentication?"
|
||||
- User asks: "Show me context around bug fixes"
|
||||
- User asks: "Timeline of database work"
|
||||
- Need to find something then see temporal context
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
# Auto mode: Uses top search result as timeline anchor
|
||||
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=auto&depth_before=10&depth_after=10"
|
||||
|
||||
# Interactive mode: Shows top N search results for manual selection
|
||||
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=interactive&limit=5"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- **query** (required): Search terms (e.g., "authentication", "bug fix", "database")
|
||||
- **mode**: Search mode
|
||||
- `auto` (default): Automatically uses top search result as timeline anchor
|
||||
- `interactive`: Returns top N search results for manual anchor selection
|
||||
- **depth_before**: Records before anchor (default: 10, max: 50) - for auto mode
|
||||
- **depth_after**: Records after anchor (default: 10, max: 50) - for auto mode
|
||||
- **limit**: Number of search results (default: 5, max: 20) - for interactive mode
|
||||
- **project**: Filter by project name (optional)
|
||||
|
||||
## Auto Mode (Recommended)
|
||||
|
||||
Automatically gets timeline around best match:
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/timeline/by-query?query=JWT+authentication&mode=auto&depth_before=10&depth_after=10"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"query": "JWT authentication",
|
||||
"mode": "auto",
|
||||
"best_match": {
|
||||
"id": 1234,
|
||||
"type": "feature",
|
||||
"title": "Implemented JWT authentication",
|
||||
"score": 0.95
|
||||
},
|
||||
"timeline": [
|
||||
// ... timeline records around observation #1234
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**When to use auto mode:**
|
||||
- You're confident the top result is what you want
|
||||
- Want fastest path to timeline context
|
||||
- Query is specific enough for accurate top result
|
||||
|
||||
## Interactive Mode
|
||||
|
||||
Shows top search results for manual review:
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=interactive&limit=5"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"query": "authentication",
|
||||
"mode": "interactive",
|
||||
"top_matches": [
|
||||
{
|
||||
"id": 1234,
|
||||
"type": "feature",
|
||||
"title": "Implemented JWT authentication",
|
||||
"subtitle": "Added token-based auth with refresh tokens",
|
||||
"score": 0.95
|
||||
},
|
||||
{
|
||||
"id": 1240,
|
||||
"type": "bugfix",
|
||||
"title": "Fixed authentication token expiration",
|
||||
"subtitle": "Resolved race condition in refresh flow",
|
||||
"score": 0.87
|
||||
}
|
||||
],
|
||||
"next_step": "Use /api/timeline/context?anchor=<id>&depth_before=10&depth_after=10"
|
||||
}
|
||||
```
|
||||
|
||||
**When to use interactive mode:**
|
||||
- Query is broad and may have multiple relevant results
|
||||
- Want to review options before getting timeline
|
||||
- Not sure which result is most relevant
|
||||
|
||||
## How to Present Results
|
||||
|
||||
**For auto mode:**
|
||||
|
||||
```markdown
|
||||
## Timeline: JWT authentication
|
||||
|
||||
**Best Match:** 🟣 Observation #1234 - Implemented JWT authentication (score: 0.95)
|
||||
|
||||
### Before (10 records)
|
||||
**2:45 PM** - 🟣 Added authentication middleware
|
||||
|
||||
### ⭐ Anchor Point (2:55 PM)
|
||||
🟣 **Observation #1234**: Implemented JWT authentication
|
||||
|
||||
### After (10 records)
|
||||
**3:00 PM** - 🎯 Session completed: JWT authentication system
|
||||
```
|
||||
|
||||
**For interactive mode:**
|
||||
|
||||
```markdown
|
||||
Found 5 matches for "authentication":
|
||||
|
||||
1. 🟣 **#1234** Implemented JWT authentication (score: 0.95)
|
||||
> Added token-based auth with refresh tokens
|
||||
|
||||
2. 🔴 **#1240** Fixed authentication token expiration (score: 0.87)
|
||||
> Resolved race condition in refresh flow
|
||||
|
||||
To see timeline context, use observation ID with timeline operation.
|
||||
```
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing query parameter:**
|
||||
```json
|
||||
{"error": "Missing required parameter: query"}
|
||||
```
|
||||
Fix: Add the query parameter
|
||||
|
||||
**No results found:**
|
||||
```json
|
||||
{"query": "foobar", "top_matches": []}
|
||||
```
|
||||
Response: "No results found for 'foobar'. Try different search terms."
|
||||
|
||||
## Tips
|
||||
|
||||
1. **Use auto mode** for specific queries: "JWT authentication implementation"
|
||||
2. **Use interactive mode** for broad queries: "authentication"
|
||||
3. Start with depth 10/10 for balanced context
|
||||
4. Be specific in queries for better auto mode accuracy
|
||||
5. This is fastest way to find + explore context in one request
|
||||
|
||||
**Token Efficiency:**
|
||||
- Auto mode: ~3,000-4,000 tokens (search + timeline)
|
||||
- Interactive mode: ~500-1,000 tokens (search results only)
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
|
||||
## Workflow Comparison
|
||||
|
||||
**timeline-by-query (auto):**
|
||||
1. One request → get timeline around best match
|
||||
2. ~3,000 tokens
|
||||
|
||||
**timeline-by-query (interactive) → timeline:**
|
||||
1. First request → see top matches (~500 tokens)
|
||||
2. Second request → get timeline for chosen match (~3,000 tokens)
|
||||
3. Total: ~3,500 tokens
|
||||
|
||||
**observations search → timeline:**
|
||||
1. Search observations (~500 tokens)
|
||||
2. Get timeline for chosen result (~3,000 tokens)
|
||||
3. Total: ~3,500 tokens
|
||||
|
||||
Use auto mode when you're confident about the query. Use interactive mode or separate search when you want more control.
|
||||
|
||||
## When to Use Timeline-by-Query
|
||||
|
||||
**Use timeline-by-query when:**
|
||||
- Need to find something AND see temporal context
|
||||
- Want one-request convenience (auto mode)
|
||||
- Investigating "what was happening when we worked on X?"
|
||||
- Don't have observation ID already
|
||||
|
||||
**Don't use timeline-by-query when:**
|
||||
- Already have observation ID (use timeline instead)
|
||||
- Just need search results (use observations search)
|
||||
- Need recent work overview (use recent-context)
|
||||
@@ -0,0 +1,174 @@
|
||||
# Get Context Timeline
|
||||
|
||||
Get a chronological timeline of observations, sessions, and prompts around a specific point in time.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks: "What was happening when we deployed?"
|
||||
- User asks: "Show me context around that bug fix"
|
||||
- User asks: "What happened before and after that change?"
|
||||
- Need temporal context around an event
|
||||
|
||||
## Command
|
||||
|
||||
```bash
|
||||
# Using observation ID as anchor
|
||||
curl -s "http://localhost:37777/api/timeline/context?anchor=1234&depth_before=10&depth_after=10"
|
||||
|
||||
# Using session ID as anchor
|
||||
curl -s "http://localhost:37777/api/timeline/context?anchor=S545&depth_before=10&depth_after=10"
|
||||
|
||||
# Using ISO timestamp as anchor
|
||||
curl -s "http://localhost:37777/api/timeline/context?anchor=2024-11-09T12:00:00Z&depth_before=10&depth_after=10"
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- **anchor** (required): Point in time to center timeline
|
||||
- Observation ID: `1234`
|
||||
- Session ID: `S545`
|
||||
- ISO timestamp: `2024-11-09T12:00:00Z`
|
||||
- **depth_before**: Number of records before anchor (default: 10, max: 50)
|
||||
- **depth_after**: Number of records after anchor (default: 10, max: 50)
|
||||
- **project**: Filter by project name (optional)
|
||||
|
||||
## Response Structure
|
||||
|
||||
Returns unified chronological timeline:
|
||||
|
||||
```json
|
||||
{
|
||||
"anchor": 1234,
|
||||
"depth_before": 10,
|
||||
"depth_after": 10,
|
||||
"total_records": 21,
|
||||
"timeline": [
|
||||
{
|
||||
"record_type": "observation",
|
||||
"id": 1230,
|
||||
"type": "feature",
|
||||
"title": "Added authentication middleware",
|
||||
"created_at_epoch": 1699564700000
|
||||
},
|
||||
{
|
||||
"record_type": "prompt",
|
||||
"id": 1250,
|
||||
"session_id": "S545",
|
||||
"prompt_preview": "How do I add JWT authentication?",
|
||||
"created_at_epoch": 1699564750000
|
||||
},
|
||||
{
|
||||
"record_type": "observation",
|
||||
"id": 1234,
|
||||
"type": "feature",
|
||||
"title": "Implemented JWT authentication",
|
||||
"created_at_epoch": 1699564800000,
|
||||
"is_anchor": true
|
||||
},
|
||||
{
|
||||
"record_type": "session",
|
||||
"id": 545,
|
||||
"session_id": "S545",
|
||||
"title": "Implemented JWT authentication system",
|
||||
"created_at_epoch": 1699564900000
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## How to Present Results
|
||||
|
||||
Present as chronological narrative with anchor highlighted:
|
||||
|
||||
```markdown
|
||||
## Timeline around Observation #1234
|
||||
|
||||
### Before (10 records)
|
||||
|
||||
**2:45 PM** - 🟣 Observation #1230: Added authentication middleware
|
||||
|
||||
**2:50 PM** - 💬 User asked: "How do I add JWT authentication?"
|
||||
|
||||
### ⭐ Anchor Point (2:55 PM)
|
||||
🟣 **Observation #1234**: Implemented JWT authentication
|
||||
|
||||
### After (10 records)
|
||||
|
||||
**3:00 PM** - 🎯 Session #545 completed: Implemented JWT authentication system
|
||||
|
||||
**3:05 PM** - 🔴 Observation #1235: Fixed token expiration edge case
|
||||
```
|
||||
|
||||
For complete formatting guidelines, see [formatting.md](formatting.md).
|
||||
|
||||
## Anchor Types
|
||||
|
||||
**Observation ID:**
|
||||
- Use when you know the specific observation ID
|
||||
- Example: `anchor=1234`
|
||||
|
||||
**Session ID:**
|
||||
- Use when you want context around a session
|
||||
- Example: `anchor=S545`
|
||||
|
||||
**ISO Timestamp:**
|
||||
- Use when you know approximate time
|
||||
- Example: `anchor=2024-11-09T14:30:00Z`
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing anchor parameter:**
|
||||
```json
|
||||
{"error": "Missing required parameter: anchor"}
|
||||
```
|
||||
Fix: Add the anchor parameter
|
||||
|
||||
**Anchor not found:**
|
||||
```json
|
||||
{"error": "Anchor not found: 9999"}
|
||||
```
|
||||
Response: "Observation #9999 not found. Check the ID or try a different anchor."
|
||||
|
||||
**Invalid timestamp:**
|
||||
```json
|
||||
{"error": "Invalid timestamp format"}
|
||||
```
|
||||
Fix: Use ISO 8601 format: `2024-11-09T14:30:00Z`
|
||||
|
||||
## Tips
|
||||
|
||||
1. Start with depth_before=10, depth_after=10 for balanced context
|
||||
2. Increase depth for broader investigation (max: 50 each)
|
||||
3. Use observation IDs from search results as anchors
|
||||
4. Timelines show all record types interleaved chronologically
|
||||
5. Perfect for understanding "what was happening when X occurred"
|
||||
|
||||
**Token Efficiency:**
|
||||
- depth 10/10: ~3,000-4,000 tokens (21 records)
|
||||
- depth 20/20: ~6,000-8,000 tokens (41 records)
|
||||
- depth 50/50: ~15,000-20,000 tokens (101 records)
|
||||
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
|
||||
|
||||
## When to Use Timeline
|
||||
|
||||
**Use timeline when:**
|
||||
- Need context around specific event
|
||||
- Understanding sequence of events
|
||||
- Investigating "what was happening then?"
|
||||
- Want all record types (observations, sessions, prompts) together
|
||||
|
||||
**Don't use timeline when:**
|
||||
- Just need recent work (use recent-context)
|
||||
- Looking for specific topics (use search)
|
||||
- Don't have an anchor point (use timeline-by-query)
|
||||
|
||||
## Comparison with Timeline-by-Query
|
||||
|
||||
| Feature | timeline | timeline-by-query |
|
||||
|---------|----------|-------------------|
|
||||
| Requires anchor | Yes (ID or timestamp) | No (uses search query) |
|
||||
| Best for | Known event investigation | Finding then exploring context |
|
||||
| Steps | 1 (direct timeline) | 2 (search + timeline) |
|
||||
| Use when | You have observation ID | You have search term |
|
||||
|
||||
Timeline is faster when you already know the anchor point.
|
||||