claude-mem/docs/hooks-architecture.mdx

# How Claude-Mem Uses Hooks: A Lifecycle-Driven Architecture

## Core Principle
**Observe the main Claude Code session from the outside, process observations in the background, inject context at the right time.**

---

## The Big Picture

Claude-Mem is fundamentally a **hook-driven system**. Every piece of functionality happens in response to lifecycle events:

```
┌─────────────────────────────────────────────────────────┐
│              CLAUDE CODE SESSION                         │
│  (Main session - user interacting with Claude)          │
│                                                          │
│  SessionStart → UserPromptSubmit → Tool Use → Stop      │
│       ↓              ↓               ↓          ↓        │
│    [Hook]         [Hook]          [Hook]     [Hook]     │
└─────────────────────────────────────────────────────────┘
       ↓              ↓               ↓          ↓
┌─────────────────────────────────────────────────────────┐
│                  CLAUDE-MEM SYSTEM                       │
│                                                          │
│  Context      New Session    Observation    Summary     │
│  Injection    Tracking       Capture        Generation  │
└─────────────────────────────────────────────────────────┘
```

**Key insight:** Claude-Mem doesn't interrupt or modify Claude Code's behavior. It observes from the outside and provides value through lifecycle hooks.

---

## Why Hooks?

### The Non-Invasive Requirement

Claude-Mem had several architectural constraints:

1. **Can't modify Claude Code**: It's a closed-source binary
2. **Must be fast**: Can't slow down the main session
3. **Must be reliable**: Can't break Claude Code if it fails
4. **Must be portable**: Works on any project without configuration

**Solution:** External command hooks configured via settings.json

### The Hook System Advantage

Claude Code's hook system provides exactly what we need:

<CardGroup cols={2}>
  <Card title="Lifecycle Events" icon="clock">
    SessionStart, UserPromptSubmit, PostToolUse, Stop
  </Card>

  <Card title="Non-Blocking" icon="forward">
    Hooks run in parallel, don't wait for completion
  </Card>

  <Card title="Context Injection" icon="upload">
    SessionStart and UserPromptSubmit can add context
  </Card>

  <Card title="Tool Observation" icon="eye">
    PostToolUse sees all tool inputs and outputs
  </Card>
</CardGroup>

---

## The Five Hooks

### Hook 1: SessionStart (Context Hook)

**Purpose:** Inject relevant context from previous sessions

**When:** Claude Code starts or resumes

**What it does:**
1. Extracts project name from current working directory
2. Queries SQLite for recent session summaries (last 10)
3. Queries SQLite for recent observations (last 50)
4. Formats as progressive disclosure index
5. Outputs to stdout (automatically injected into context)

**Configuration:**
```json
{
  "hooks": {
    "SessionStart": [{
      "matcher": "startup",
      "hooks": [{
        "type": "command",
        "command": "${CLAUDE_PLUGIN_ROOT}/scripts/context-hook.js",
        "timeout": 120
      }]
    }]
  }
}
```

**Key decisions:**
- ✅ Only runs on "startup" (not "clear" or "compact")
- ✅ 120-second timeout for npm install (v4.3.1 fix)
- ✅ Uses `--loglevel=silent` for clean JSON output
- ✅ Progressive disclosure format (index, not full details)

**Output format:**
```markdown
# [claude-mem] recent context

**Legend:** 🎯 session-request | 🔴 gotcha | 🟡 problem-solution ...

### Oct 26, 2025

**General**
| ID | Time | T | Title | Tokens |
|----|------|---|-------|--------|
| #2586 | 12:58 AM | 🔵 | Context hook file empty | ~51 |

*Use claude-mem MCP search to access full details*
```

**Source:** `src/hooks/context-hook.ts` → `plugin/scripts/context-hook.js`

---

### Hook 2: UserPromptSubmit (New Session Hook)

**Purpose:** Initialize session tracking when user submits a prompt

**When:** Before Claude processes the user's message

**What it does:**
1. Reads user prompt and session ID from stdin
2. Creates new session record in SQLite
3. Saves raw user prompt for full-text search (v4.2.0+)
4. Starts PM2 worker service if not running
5. Returns immediately (non-blocking)

**Configuration:**
```json
{
  "hooks": {
    "UserPromptSubmit": [{
      "hooks": [{
        "type": "command",
        "command": "${CLAUDE_PLUGIN_ROOT}/scripts/new-hook.js"
      }]
    }]
  }
}
```

**Key decisions:**
- ✅ No matcher (runs for all prompts)
- ✅ Creates session record immediately
- ✅ Stores raw prompts for search (privacy note: local SQLite only)
- ✅ Auto-starts worker service
- ✅ Suppresses output (`suppressOutput: true`)

**Database operations:**
```sql
INSERT INTO sdk_sessions (claude_session_id, project, user_prompt, ...)
VALUES (?, ?, ?, ...)

INSERT INTO user_prompts (session_id, prompt, prompt_number, ...)
VALUES (?, ?, ?, ...)
```

**Source:** `src/hooks/new-hook.ts` → `plugin/scripts/new-hook.js`

---

### Hook 3: PostToolUse (Save Observation Hook)

**Purpose:** Capture tool execution observations for later processing

**When:** Immediately after any tool completes successfully

**What it does:**
1. Receives tool name, input, output from stdin
2. Finds active session for current project
3. Enqueues observation in observation_queue table
4. Returns immediately (processing happens in worker)

**Configuration:**
```json
{
  "hooks": {
    "PostToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "${CLAUDE_PLUGIN_ROOT}/scripts/save-hook.js"
      }]
    }]
  }
}
```

**Key decisions:**
- ✅ Matcher: `*` (captures all tools)
- ✅ Non-blocking (just enqueues, doesn't process)
- ✅ Worker processes observations asynchronously
- ✅ Parallel execution safe (each hook gets own stdin)

**Database operations:**
```sql
INSERT INTO observation_queue (session_id, tool_name, tool_input, tool_output, ...)
VALUES (?, ?, ?, ?, ...)
```

**What gets queued:**
```json
{
  "session_id": "abc123",
  "tool_name": "Edit",
  "tool_input": {
    "file_path": "/path/to/file.ts",
    "old_string": "...",
    "new_string": "..."
  },
  "tool_output": {
    "success": true,
    "linesChanged": 5
  },
  "created_at_epoch": 1698765432
}
```

**Source:** `src/hooks/save-hook.ts` → `plugin/scripts/save-hook.js`

---

### Hook 4: Summary Hook (Mid-Session Checkpoint)

**Purpose:** Generate AI-powered session summaries during the session

**When:** Triggered programmatically by the worker service

**What it does:**
1. Gathers session observations from database
2. Sends to Claude Agent SDK for summarization
3. Processes response and extracts structured summary
4. Stores in session_summaries table

**Configuration:**
```json
{
  "hooks": {
    "Summary": [{
      "hooks": [{
        "type": "command",
        "command": "${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js"
      }]
    }]
  }
}
```

**Key decisions:**
- ✅ Triggered by worker, not by Claude Code lifecycle
- ✅ Multiple summaries per session (v4.2.0+)
- ✅ Summaries are checkpoints, not endings
- ✅ Uses Claude Agent SDK for AI compression

**Summary structure:**
```xml
<summary>
  <request>User's original request</request>
  <investigated>What was examined</investigated>
  <learned>Key discoveries</learned>
  <completed>Work finished</completed>
  <next_steps>Remaining tasks</next_steps>
  <files_read>
    <file>path/to/file1.ts</file>
    <file>path/to/file2.ts</file>
  </files_read>
  <files_modified>
    <file>path/to/file3.ts</file>
  </files_modified>
  <notes>Additional context</notes>
</summary>
```

**Source:** `src/hooks/summary-hook.ts` → `plugin/scripts/summary-hook.js`

---

### Hook 5: SessionEnd (Cleanup Hook)

**Purpose:** Mark sessions as completed when they end

**When:** Claude Code session ends (not on `/clear`)

**What it does:**
1. Marks session as completed in database
2. Allows worker to finish processing
3. Performs graceful cleanup

**Configuration:**
```json
{
  "hooks": {
    "SessionEnd": [{
      "hooks": [{
        "type": "command",
        "command": "${CLAUDE_PLUGIN_ROOT}/scripts/cleanup-hook.js"
      }]
    }]
  }
}
```

**Key decisions:**
- ✅ Graceful completion (v4.1.0+)
- ✅ No longer sends DELETE to workers
- ✅ Skips cleanup on `/clear` commands
- ✅ Preserves ongoing sessions

**Why graceful cleanup?**

**Old approach (v3):**
```typescript
// ❌ Aggressive cleanup
SessionEnd → DELETE /worker/session → Worker stops immediately
```

**Problems:**
- Interrupted summary generation
- Lost pending observations
- Race conditions

**New approach (v4.1.0+):**
```typescript
// ✅ Graceful completion
SessionEnd → UPDATE sessions SET completed_at = NOW()
Worker sees completion → Finishes processing → Exits naturally
```

**Benefits:**
- Worker finishes important operations
- Summaries complete successfully
- Clean state transitions

**Source:** `src/hooks/cleanup-hook.ts` → `plugin/scripts/cleanup-hook.js`

---

## Hook Execution Flow

### Session Lifecycle

```mermaid
sequenceDiagram
    participant User
    participant Claude
    participant Hooks
    participant Worker
    participant DB

    User->>Claude: Start Claude Code
    Claude->>Hooks: SessionStart hook
    Hooks->>DB: Query recent context
    DB-->>Hooks: Session summaries + observations
    Hooks-->>Claude: Inject context
    Note over Claude: Context available for session

    User->>Claude: Submit prompt
    Claude->>Hooks: UserPromptSubmit hook
    Hooks->>DB: Create session record
    Hooks->>Worker: Start worker (if not running)
    Worker-->>DB: Ready to process

    Claude->>Claude: Execute tools
    Claude->>Hooks: PostToolUse (multiple times)
    Hooks->>DB: Queue observations
    Note over Worker: Polls queue, processes observations

    Worker->>Worker: AI compression
    Worker->>DB: Store compressed observations
    Worker->>Hooks: Trigger summary hook
    Hooks->>DB: Store session summary

    User->>Claude: Finish
    Claude->>Hooks: SessionEnd hook
    Hooks->>DB: Mark session complete
    Worker->>DB: Check completion
    Worker->>Worker: Finish processing
    Worker->>Worker: Exit gracefully
```

### Hook Timing

| Event | Timing | Blocking | Timeout | Output Handling |
|-------|--------|----------|---------|-----------------|
| **SessionStart** | Before session | No | 120s | stdout → context |
| **UserPromptSubmit** | Before processing | No | 60s | stdout → context |
| **PostToolUse** | After tool | No | 60s | Transcript only |
| **Summary** | Worker triggered | No | 300s | Database |
| **SessionEnd** | On exit | No | 60s | Log only |

---

## The Worker Service Architecture

### Why a Background Worker?

**Problem:** Hooks must be fast (< 1 second)

**Reality:** AI compression takes 5-30 seconds per observation

**Solution:** Hooks enqueue observations, worker processes async

```
┌─────────────────────────────────────────────────────────┐
│                   HOOK (Fast)                            │
│  1. Read stdin (< 1ms)                                  │
│  2. Insert into queue (< 10ms)                          │
│  3. Return success (< 20ms total)                       │
└─────────────────────────────────────────────────────────┘
                        ↓ (queue)
┌─────────────────────────────────────────────────────────┐
│                 WORKER (Slow)                            │
│  1. Poll queue every 1s                                 │
│  2. Process observation via Claude SDK (5-30s)          │
│  3. Parse and store results                             │
│  4. Mark observation processed                          │
└─────────────────────────────────────────────────────────┘
```

### PM2 Process Management

**Technology:** PM2 (process manager for Node.js)

**Why PM2:**
- Auto-restart on failure
- Log management
- Process monitoring
- Cross-platform (works on macOS, Linux, Windows)
- No systemd/launchd needed

**Configuration:**
```javascript
// ecosystem.config.cjs
module.exports = {
  apps: [{
    name: 'claude-mem-worker',
    script: './plugin/scripts/worker-service.cjs',
    instances: 1,
    autorestart: true,
    watch: false,
    max_memory_restart: '500M',
    env: {
      NODE_ENV: 'production',
      CLAUDE_MEM_WORKER_PORT: 37777
    }
  }]
};
```

**Worker lifecycle:**
```bash
# Started by new-hook (if not running)
pm2 start ecosystem.config.cjs

# Status check
pm2 status claude-mem-worker

# View logs
pm2 logs claude-mem-worker

# Restart
pm2 restart claude-mem-worker
```

### Worker HTTP API

**Technology:** Express.js REST API on port 37777

**Endpoints:**

| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/health` | GET | Health check |
| `/sessions` | POST | Create session |
| `/sessions/:id` | GET | Get session status |
| `/sessions/:id` | PATCH | Update session |
| `/observations` | POST | Enqueue observation |
| `/observations/:id` | GET | Get observation |

**Why HTTP API?**
- Language-agnostic (hooks can be any language)
- Easy debugging (curl commands)
- Standard error handling
- Proper async handling

---

## Design Patterns

### Pattern 1: Fire-and-Forget Hooks

**Principle:** Hooks should return immediately, not wait for completion

```typescript
// ❌ Bad: Hook waits for processing
export async function saveHook(stdin: HookInput) {
  const observation = parseInput(stdin);
  await processObservation(observation);  // BLOCKS!
  return success();
}

// ✅ Good: Hook enqueues and returns
export async function saveHook(stdin: HookInput) {
  const observation = parseInput(stdin);
  await enqueueObservation(observation);  // Fast
  return success();  // Immediate
}
```

### Pattern 2: Queue-Based Processing

**Principle:** Decouple capture from processing

```
Hook (capture) → Queue (buffer) → Worker (process)
```

**Benefits:**
- Parallel hook execution safe
- Worker failure doesn't affect hooks
- Retry logic centralized
- Backpressure handling

### Pattern 3: Graceful Degradation

**Principle:** Memory system failure shouldn't break Claude Code

```typescript
try {
  await captureObservation();
} catch (error) {
  // Log error, but don't throw
  console.error('Memory capture failed:', error);
  return { continue: true, suppressOutput: true };
}
```

**Failure modes:**
- Database locked → Skip observation, log error
- Worker crashed → Auto-restart via PM2
- Network issue → Retry with exponential backoff
- Disk full → Warn user, disable memory

### Pattern 4: Progressive Enhancement

**Principle:** Core functionality works without memory, memory enhances it

```
Without memory: Claude Code works normally
With memory:    Claude Code + context from past sessions
Memory broken:  Falls back to working normally
```

---

## Hook Debugging

### Debug Mode

Enable detailed hook execution logs:

```bash
claude --debug
```

**Output:**
```
[DEBUG] Executing hooks for PostToolUse:Write
[DEBUG] Getting matching hook commands for PostToolUse with query: Write
[DEBUG] Found 1 hook matchers in settings
[DEBUG] Matched 1 hooks for query "Write"
[DEBUG] Found 1 hook commands to execute
[DEBUG] Executing hook command: ${CLAUDE_PLUGIN_ROOT}/scripts/save-hook.js with timeout 60000ms
[DEBUG] Hook command completed with status 0: {"continue":true,"suppressOutput":true}
```

### Common Issues

<AccordionGroup>
  <Accordion title="Hook not executing">
    **Symptoms:** Hook command never runs

    **Debugging:**
    1. Check `/hooks` menu - is hook registered?
    2. Verify matcher pattern (case-sensitive!)
    3. Test command manually: `echo '{}' | node save-hook.js`
    4. Check file permissions (executable?)
  </Accordion>

  <Accordion title="Hook times out">
    **Symptoms:** Hook execution exceeds timeout

    **Debugging:**
    1. Check timeout setting (default 60s)
    2. Identify slow operation (database? network?)
    3. Move slow operation to worker
    4. Increase timeout if necessary
  </Accordion>

  <Accordion title="Context not injecting">
    **Symptoms:** SessionStart hook runs but context missing

    **Debugging:**
    1. Check stdout (must be valid JSON or plain text)
    2. Verify no stderr output (pollutes JSON)
    3. Check exit code (must be 0)
    4. Look for npm install output (v4.3.1 fix)
  </Accordion>

  <Accordion title="Observations not captured">
    **Symptoms:** PostToolUse hook runs but observations missing

    **Debugging:**
    1. Check database: `sqlite3 ~/.claude-mem/claude-mem.db "SELECT * FROM observation_queue"`
    2. Verify session exists: `SELECT * FROM sdk_sessions`
    3. Check worker status: `pm2 status`
    4. View worker logs: `pm2 logs claude-mem-worker`
  </Accordion>
</AccordionGroup>

### Testing Hooks Manually

```bash
# Test context hook
echo '{
  "session_id": "test123",
  "cwd": "/Users/alex/projects/my-app",
  "hook_event_name": "SessionStart",
  "source": "startup"
}' | node plugin/scripts/context-hook.js

# Test save hook
echo '{
  "session_id": "test123",
  "tool_name": "Edit",
  "tool_input": {"file_path": "test.ts"},
  "tool_output": {"success": true}
}' | node plugin/scripts/save-hook.js

# Test with actual Claude Code
claude --debug
/hooks  # View registered hooks
# Submit prompt and watch debug output
```

---

## Performance Considerations

### Hook Execution Time

**Target:** < 100ms per hook

**Actual measurements:**

| Hook | Average | p95 | p99 |
|------|---------|-----|-----|
| SessionStart | 45ms | 120ms | 250ms |
| UserPromptSubmit | 12ms | 25ms | 50ms |
| PostToolUse | 8ms | 15ms | 30ms |
| SessionEnd | 5ms | 10ms | 20ms |

**Why SessionStart is slower:**
- npm install check (idempotent but runs every time)
- Database query for 10 sessions + 50 observations
- Formatting progressive disclosure index

**Optimization (v4.3.1):**
- Use `--loglevel=silent` for npm install
- Cache package.json hash to skip unnecessary installs
- Use prepared statements for database queries

### Database Performance

**Schema optimizations:**
- Indexes on `project`, `created_at_epoch`, `claude_session_id`
- FTS5 virtual tables for full-text search
- WAL mode for concurrent reads/writes

**Query patterns:**
```sql
-- Fast: Uses index on (project, created_at_epoch)
SELECT * FROM session_summaries
WHERE project = ?
ORDER BY created_at_epoch DESC
LIMIT 10

-- Fast: Uses index on claude_session_id
SELECT * FROM sdk_sessions
WHERE claude_session_id = ?
LIMIT 1

-- Fast: FTS5 full-text search
SELECT * FROM observations_fts
WHERE observations_fts MATCH ?
ORDER BY rank
LIMIT 20
```

### Worker Throughput

**Bottleneck:** Claude API latency (5-30s per observation)

**Mitigation:**
- Process observations sequentially (simpler, more predictable)
- Skip low-value observations (TodoWrite, ListMcpResourcesTool)
- Batch summaries (generate every N observations, not every observation)

**Future optimization:**
- Parallel processing (multiple workers)
- Smart batching (combine related observations)
- Lazy summarization (summarize only when needed)

---

## Security Considerations

### Hook Command Safety

**Risk:** Hooks execute arbitrary commands with user permissions

**Mitigations:**
1. **Frozen at startup:** Hook configuration captured at start, changes require review
2. **User review required:** `/hooks` menu shows changes, requires approval
3. **Plugin isolation:** `${CLAUDE_PLUGIN_ROOT}` prevents path traversal
4. **Input validation:** Hooks validate stdin schema before processing

### Data Privacy

**What gets stored:**
- User prompts (raw text) - v4.2.0+
- Tool inputs and outputs
- File paths read/modified
- Session summaries

**Privacy guarantees:**
- All data stored locally in `~/.claude-mem/claude-mem.db`
- No cloud uploads (API calls only for AI compression)
- SQLite file permissions: user-only read/write
- No analytics or telemetry

### API Key Protection

**Configuration:**
- Anthropic API key in `~/.anthropic/api_key` or `ANTHROPIC_API_KEY` env var
- Worker inherits environment from Claude Code
- Never logged or stored in database

---

## Key Takeaways

1. **Hooks are interfaces**: They define clean boundaries between systems
2. **Non-blocking is critical**: Hooks must return fast, workers do the heavy lifting
3. **Graceful degradation**: Memory system can fail without breaking Claude Code
4. **Queue-based decoupling**: Capture and processing happen independently
5. **Progressive disclosure**: Context injection uses index-first approach
6. **Lifecycle alignment**: Each hook has a clear, single purpose

---

## Further Reading

- [Claude Code Hooks Reference](https://docs.claude.com/claude-code/hooks) - Official documentation
- [Progressive Disclosure](/docs/progressive-disclosure) - Context priming philosophy
- [Architecture Evolution](/docs/architecture-evolution) - v3 to v4 journey
- [Worker Service Design](/docs/worker-service) - Background processing details

---

*The hook-driven architecture enables Claude-Mem to be both powerful and invisible. Users never notice the memory system working - it just makes Claude smarter over time.*