12603a1a5c
Add comprehensive technical documentation explaining the v7.1.0 migration from PM2 to Bun-based process management and better-sqlite3 to bun:sqlite database driver. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
552 lines
17 KiB
Plaintext
552 lines
17 KiB
Plaintext
---
|
|
title: "PM2 to Bun Migration"
|
|
description: "Complete technical documentation for the process management and database driver migration in v7.1.0"
|
|
---
|
|
|
|
# PM2 to Bun Migration: Complete Technical Documentation
|
|
|
|
**Version**: 7.1.0
|
|
**Date**: December 2025
|
|
**Migration Type**: Process Management (PM2 → Bun) + Database Driver (better-sqlite3 → bun:sqlite)
|
|
|
|
## Executive Summary
|
|
|
|
Claude-mem version 7.1.0 introduces two major architectural migrations:
|
|
|
|
1. **Process Management**: PM2 → Custom Bun-based ProcessManager
|
|
2. **Database Driver**: better-sqlite3 npm package → bun:sqlite runtime module
|
|
|
|
Both migrations are **automatic** and **transparent** to end users. The first time a hook fires after updating to 7.1.0+, the system performs a one-time cleanup of legacy PM2 processes and transitions to the new architecture.
|
|
|
|
### Key Benefits
|
|
|
|
- **Simplified Dependencies**: Removes PM2 and better-sqlite3 npm packages
|
|
- **Improved Cross-Platform Support**: Better Windows compatibility
|
|
- **Faster Installation**: No native module compilation required
|
|
- **Built-in Runtime**: Leverages Bun's built-in process management and SQLite
|
|
- **Reduced Complexity**: Custom ProcessManager is simpler than PM2 integration
|
|
|
|
### Migration Impact
|
|
|
|
- **Data Preservation**: User data, settings, and database remain unchanged
|
|
- **Automatic Cleanup**: Old PM2 processes automatically terminated (all platforms)
|
|
- **No User Action Required**: Migration happens automatically on first hook trigger
|
|
- **Backward Compatible**: SQLite database format unchanged (only driver changed)
|
|
|
|
## Architecture Comparison
|
|
|
|
### Old System (PM2-based)
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Process Management (PM2)">
|
|
**Component**: PM2 (Process Manager 2)
|
|
- **Package**: `pm2` npm dependency
|
|
- **Process Name**: `claude-mem-worker`
|
|
- **Management**: External PM2 daemon manages lifecycle
|
|
- **Discovery**: `pm2 list`, `pm2 describe` commands
|
|
- **Auto-restart**: PM2 automatically restarts on crash
|
|
- **Logs**: `~/.pm2/logs/claude-mem-worker-*.log`
|
|
- **PID File**: `~/.pm2/pids/claude-mem-worker.pid`
|
|
|
|
**Lifecycle Commands**:
|
|
```bash
|
|
pm2 start <script> # Start worker
|
|
pm2 stop claude-mem-worker # Stop worker
|
|
pm2 restart claude-mem-worker # Restart worker
|
|
pm2 delete claude-mem-worker # Remove from PM2
|
|
pm2 logs claude-mem-worker # View logs
|
|
```
|
|
|
|
**Pain Points**:
|
|
- Additional npm dependency required
|
|
- PM2 daemon must be running
|
|
- Potential conflicts with other PM2 processes
|
|
- Windows compatibility issues
|
|
- Complex configuration for simple use case
|
|
</Accordion>
|
|
|
|
<Accordion title="Database Driver (better-sqlite3)">
|
|
**Component**: better-sqlite3
|
|
- **Package**: `better-sqlite3` npm package (native module)
|
|
- **Installation**: Requires native compilation (node-gyp)
|
|
- **Windows**: Requires Visual Studio build tools + Python
|
|
- **Import**: `import Database from 'better-sqlite3'`
|
|
|
|
**Installation Requirements**:
|
|
- Node.js development headers
|
|
- C++ compiler (gcc/clang on Mac/Linux, MSVC on Windows)
|
|
- Python (for node-gyp)
|
|
- Windows: Visual Studio Build Tools
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
### New System (Bun-based)
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Process Management (Custom ProcessManager)">
|
|
**Component**: Custom ProcessManager (`src/services/process/ProcessManager.ts`)
|
|
- **Package**: Built-in Bun APIs (no external dependency)
|
|
- **Process Spawn**: `Bun.spawn()` with detached mode
|
|
- **Management**: Direct process control via PID file
|
|
- **Discovery**: PID file + process existence check + HTTP health check
|
|
- **Auto-restart**: Hook-triggered restart on failure detection
|
|
- **Logs**: `~/.claude-mem/logs/worker-YYYY-MM-DD.log`
|
|
- **PID File**: `~/.claude-mem/.worker.pid`
|
|
- **Port File**: `~/.claude-mem/.worker.port` (new)
|
|
|
|
**Lifecycle Commands**:
|
|
```bash
|
|
npm run worker:start # Start worker
|
|
npm run worker:stop # Stop worker
|
|
npm run worker:restart # Restart worker
|
|
npm run worker:status # Check status
|
|
npm run worker:logs # View logs
|
|
```
|
|
|
|
**Core Mechanisms**:
|
|
|
|
1. **PID File Management**:
|
|
- File: `~/.claude-mem/.worker.pid`
|
|
- Content: Process ID (e.g., "35557")
|
|
- Validation: Process existence via `kill(pid, 0)` signal
|
|
|
|
2. **Port File Management**:
|
|
- File: `~/.claude-mem/.worker.port`
|
|
- Content: Two lines (port number, PID)
|
|
- Purpose: Track port binding and validate PID match
|
|
|
|
3. **Health Checking**:
|
|
- Layer 1: PID file exists?
|
|
- Layer 2: Process alive? (`kill(pid, 0)`)
|
|
- Layer 3: HTTP health check (`GET /health`)
|
|
- All three must pass for "healthy" status
|
|
|
|
**Advantages**:
|
|
- No external dependencies
|
|
- Simpler codebase (direct control)
|
|
- Better error handling and validation
|
|
- Platform-agnostic (Bun handles platform differences)
|
|
</Accordion>
|
|
|
|
<Accordion title="Database Driver (bun:sqlite)">
|
|
**Component**: bun:sqlite
|
|
- **Package**: Built into Bun runtime (no npm package)
|
|
- **Installation**: None required (comes with Bun ≥1.0)
|
|
- **Platform**: Works anywhere Bun works
|
|
- **Import**: `import { Database } from 'bun:sqlite'`
|
|
- **API**: Similar to better-sqlite3 (synchronous)
|
|
|
|
**Installation Requirements**:
|
|
- Bun ≥1.0 (automatically installed if missing)
|
|
- No native compilation required
|
|
- No platform-specific build tools needed
|
|
|
|
**Compatibility**:
|
|
- SQLite database format: **Unchanged**
|
|
- Database file: `~/.claude-mem/claude-mem.db` (same location)
|
|
- Query syntax: **Identical** (both use SQLite SQL)
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
## Migration Mechanics
|
|
|
|
### One-Time PM2 Cleanup
|
|
|
|
The migration system uses a marker-based approach to perform PM2 cleanup exactly once.
|
|
|
|
**Implementation**: `src/shared/worker-utils.ts:73-86`
|
|
|
|
```typescript
|
|
// Clean up legacy PM2 (one-time migration)
|
|
const pm2MigratedMarker = join(DATA_DIR, '.pm2-migrated');
|
|
|
|
if (!existsSync(pm2MigratedMarker)) {
|
|
try {
|
|
spawnSync('pm2', ['delete', 'claude-mem-worker'], { stdio: 'ignore' });
|
|
// Mark migration as complete
|
|
writeFileSync(pm2MigratedMarker, new Date().toISOString(), 'utf-8');
|
|
logger.debug('SYSTEM', 'PM2 cleanup completed and marked');
|
|
} catch {
|
|
// PM2 not installed or process doesn't exist - still mark as migrated
|
|
writeFileSync(pm2MigratedMarker, new Date().toISOString(), 'utf-8');
|
|
}
|
|
}
|
|
```
|
|
|
|
### Migration Trigger Points
|
|
|
|
<Steps>
|
|
<Step title="Hook Execution">
|
|
SessionStart, UserPromptSubmit, or PostToolUse hooks execute using new 7.1.0 code
|
|
</Step>
|
|
<Step title="Worker Status Check">
|
|
`ensureWorkerRunning()` checks if `~/.claude-mem/.worker.pid` exists (it doesn't for first run after update)
|
|
</Step>
|
|
<Step title="Start Worker Decision">
|
|
Worker not running → Call `startWorker()`
|
|
</Step>
|
|
<Step title="Migration Check">
|
|
Check if `~/.claude-mem/.pm2-migrated` exists
|
|
</Step>
|
|
<Step title="PM2 Cleanup">
|
|
Execute `pm2 delete claude-mem-worker` (errors ignored), create marker file
|
|
</Step>
|
|
<Step title="New Worker Start">
|
|
Spawn new Bun-managed worker process with PID and port files
|
|
</Step>
|
|
</Steps>
|
|
|
|
### Marker File
|
|
|
|
**Location**: `~/.claude-mem/.pm2-migrated`
|
|
|
|
**Content**: ISO 8601 timestamp
|
|
```
|
|
2025-12-13T00:18:39.673Z
|
|
```
|
|
|
|
**Purpose**:
|
|
- One-time migration flag
|
|
- Prevents repeated PM2 cleanup on every start
|
|
- Persists across restarts and reboots
|
|
|
|
**Lifecycle**:
|
|
- Created: First hook trigger after update to 7.1.0+ (all platforms)
|
|
- Updated: Never
|
|
- Deleted: Never (user could manually delete to force re-migration)
|
|
|
|
## User Experience Timeline
|
|
|
|
### First Session After Update
|
|
|
|
<Note>
|
|
This is the critical migration moment. The process takes approximately 2-5 seconds.
|
|
</Note>
|
|
|
|
**Step-by-Step Execution**:
|
|
|
|
1. **Hook fires** (SessionStart most common)
|
|
2. **Worker status check**: No PID file → worker not running
|
|
3. **Migration check**: No marker file → run PM2 cleanup
|
|
4. **PM2 cleanup**: `pm2 delete claude-mem-worker` (old worker terminated)
|
|
5. **Marker creation**: `~/.claude-mem/.pm2-migrated` with timestamp
|
|
6. **New worker start**: Bun process spawned, PID/port files created
|
|
7. **Verification**: Process check + HTTP health check
|
|
8. **Hook completes**: Claude Code session starts normally
|
|
|
|
**User Observable Behavior**:
|
|
- Slight delay on first startup (PM2 cleanup + new worker spawn)
|
|
- No error messages (cleanup failures silently handled)
|
|
- Worker appears running via `npm run worker:status`
|
|
- Old PM2 worker no longer in `pm2 list`
|
|
|
|
### Subsequent Sessions
|
|
|
|
After migration completes, every hook trigger follows the fast path:
|
|
|
|
1. PID file exists? **YES**
|
|
2. Process alive? **YES**
|
|
3. HTTP health check? **SUCCESS**
|
|
4. Result: Worker already running, done (~50ms)
|
|
|
|
No migration logic runs on subsequent sessions.
|
|
|
|
## Platform-Specific Behavior
|
|
|
|
### Platform Comparison
|
|
|
|
| Feature | macOS | Linux | Windows |
|
|
|---------|-------|-------|---------|
|
|
| PM2 Cleanup | Attempted | Attempted | Attempted |
|
|
| Marker File | Created | Created | Created |
|
|
| Process Signals | POSIX (native) | POSIX (native) | Bun abstraction |
|
|
| Bun Support | Full | Full | Full |
|
|
| PID File | Yes | Yes | Yes |
|
|
| Port File | Yes | Yes | Yes |
|
|
| Health Check | HTTP | HTTP | HTTP |
|
|
| Migration Delay | ~2-5s first time | ~2-5s first time | ~2-5s first time |
|
|
|
|
### Platform Notes
|
|
|
|
<Tabs>
|
|
<Tab title="macOS">
|
|
- POSIX signal handling works natively
|
|
- Bun fully supported
|
|
- No platform-specific workarounds needed
|
|
</Tab>
|
|
<Tab title="Linux">
|
|
- Identical behavior to macOS
|
|
- POSIX signal handling
|
|
- Works on Ubuntu, Debian, RHEL, CentOS, Arch
|
|
- Alpine may require glibc (not musl)
|
|
</Tab>
|
|
<Tab title="Windows">
|
|
- PM2 cleanup now runs (safe due to try/catch)
|
|
- Bun abstracts signal handling differences
|
|
- Path module handles Windows separators
|
|
- File locking handled by SQLite
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
## Observable Changes
|
|
|
|
### Command Changes
|
|
|
|
| Old (PM2) | New (Bun) | Notes |
|
|
|-----------|-----------|-------|
|
|
| `pm2 list` | `npm run worker:status` | Shows worker status |
|
|
| `pm2 start <script>` | `npm run worker:start` | Start worker |
|
|
| `pm2 stop claude-mem-worker` | `npm run worker:stop` | Stop worker |
|
|
| `pm2 restart claude-mem-worker` | `npm run worker:restart` | Restart worker |
|
|
| `pm2 delete claude-mem-worker` | `npm run worker:stop` | Remove worker |
|
|
| `pm2 logs claude-mem-worker` | `npm run worker:logs` | View logs |
|
|
| `pm2 describe claude-mem-worker` | `npm run worker:status` | Detailed status |
|
|
| `pm2 monit` | No equivalent | PM2-specific monitoring |
|
|
|
|
### File Location Changes
|
|
|
|
**Logs**:
|
|
```
|
|
Old: ~/.pm2/logs/claude-mem-worker-out.log
|
|
~/.pm2/logs/claude-mem-worker-error.log
|
|
|
|
New: ~/.claude-mem/logs/worker-YYYY-MM-DD.log
|
|
```
|
|
|
|
**PID Files**:
|
|
```
|
|
Old: ~/.pm2/pids/claude-mem-worker.pid
|
|
|
|
New: ~/.claude-mem/.worker.pid
|
|
```
|
|
|
|
**Process State**:
|
|
```
|
|
Old: PM2 daemon memory (pm2 save)
|
|
|
|
New: ~/.claude-mem/.worker.pid
|
|
~/.claude-mem/.worker.port
|
|
~/.claude-mem/.pm2-migrated (all platforms)
|
|
```
|
|
|
|
**Database** (unchanged):
|
|
```
|
|
Same: ~/.claude-mem/claude-mem.db
|
|
```
|
|
|
|
### User-Visible Changes
|
|
|
|
**Before Update**:
|
|
```bash
|
|
$ pm2 list
|
|
┌────┬────────────────────┬─────────┬─────────┬──────────┐
|
|
│ id │ name │ status │ restart │ uptime │
|
|
├────┼────────────────────┼─────────┼─────────┼──────────┤
|
|
│ 0 │ claude-mem-worker │ online │ 0 │ 2d 5h │
|
|
└────┴────────────────────┴─────────┴─────────┴──────────┘
|
|
```
|
|
|
|
**After Update**:
|
|
```bash
|
|
$ pm2 list
|
|
# Empty - worker no longer managed by PM2
|
|
|
|
$ npm run worker:status
|
|
Worker is running
|
|
PID: 35557
|
|
Port: 37777
|
|
Uptime: 2h 15m
|
|
```
|
|
|
|
### Orphaned Files
|
|
|
|
After migration, these PM2 files may remain (safe to delete):
|
|
|
|
```
|
|
~/.pm2/ # Entire PM2 directory
|
|
~/.pm2/logs/ # Old logs
|
|
~/.pm2/pids/ # Old PID files
|
|
~/.pm2/pm2.log # PM2 daemon log
|
|
~/.pm2/dump.pm2 # PM2 process dump
|
|
```
|
|
|
|
**Cleanup (optional)**:
|
|
```bash
|
|
# Remove PM2 entirely (if not used for other processes)
|
|
pm2 kill
|
|
rm -rf ~/.pm2
|
|
|
|
# Or just remove claude-mem logs
|
|
rm -f ~/.pm2/logs/claude-mem-worker-*.log
|
|
rm -f ~/.pm2/pids/claude-mem-worker.pid
|
|
```
|
|
|
|
## File System State
|
|
|
|
### State Directory Structure
|
|
|
|
**Before Migration** (PM2 system):
|
|
```
|
|
~/.claude-mem/
|
|
├── claude-mem.db # Database (unchanged)
|
|
├── chroma/ # Vector embeddings (unchanged)
|
|
├── logs/ # Application logs (unchanged)
|
|
└── settings.json # User settings (unchanged)
|
|
|
|
~/.pm2/
|
|
├── logs/
|
|
│ ├── claude-mem-worker-out.log
|
|
│ └── claude-mem-worker-error.log
|
|
├── pids/
|
|
│ └── claude-mem-worker.pid
|
|
└── pm2.log
|
|
```
|
|
|
|
**After Migration** (Bun system):
|
|
```
|
|
~/.claude-mem/
|
|
├── claude-mem.db # Database (same file)
|
|
├── chroma/ # Vector embeddings (unchanged)
|
|
├── logs/
|
|
│ └── worker-2025-12-13.log # New log format
|
|
├── settings.json # User settings (unchanged)
|
|
├── .worker.pid # NEW: Process ID
|
|
├── .worker.port # NEW: Port + PID
|
|
└── .pm2-migrated # NEW: Migration marker (all platforms)
|
|
|
|
~/.pm2/ # Orphaned (safe to delete)
|
|
├── logs/ # Old logs (no longer written)
|
|
├── pids/ # Old PID (no longer updated)
|
|
└── pm2.log # PM2 daemon log (not used)
|
|
```
|
|
|
|
## Edge Cases and Troubleshooting
|
|
|
|
### Scenario 1: Migration Fails (PM2 Still Running)
|
|
|
|
<Warning>
|
|
This is rare but can happen if PM2 has watch mode enabled or the process is manually restarted.
|
|
</Warning>
|
|
|
|
**Symptoms**:
|
|
- `pm2 list` still shows `claude-mem-worker`
|
|
- Port conflict errors in logs
|
|
- Worker fails to start
|
|
|
|
**Resolution**:
|
|
```bash
|
|
# Manual cleanup
|
|
pm2 delete claude-mem-worker
|
|
pm2 save # Persist the deletion
|
|
|
|
# Force re-migration (optional)
|
|
rm ~/.claude-mem/.pm2-migrated
|
|
|
|
# Restart worker
|
|
npm run worker:restart
|
|
```
|
|
|
|
### Scenario 2: Stale PID File (Process Dead)
|
|
|
|
**Symptoms**:
|
|
- `npm run worker:status` shows "not running"
|
|
- `.worker.pid` file exists
|
|
- Process ID doesn't exist
|
|
|
|
**Automatic Recovery**: Next hook trigger detects dead process and starts a fresh worker.
|
|
|
|
**Manual Resolution**:
|
|
```bash
|
|
rm ~/.claude-mem/.worker.pid
|
|
rm ~/.claude-mem/.worker.port
|
|
npm run worker:start
|
|
```
|
|
|
|
### Scenario 3: Port Already in Use
|
|
|
|
**Error**: `EADDRINUSE: address already in use`
|
|
|
|
**Resolution**:
|
|
```bash
|
|
# Check what's using the port
|
|
lsof -i :37777
|
|
|
|
# Kill the process
|
|
kill -9 <PID>
|
|
|
|
# Restart worker
|
|
npm run worker:restart
|
|
```
|
|
|
|
### Common Error Messages
|
|
|
|
| Error | Cause | Resolution |
|
|
|-------|-------|------------|
|
|
| `EADDRINUSE` | Port already in use | `lsof -i :37777` then kill conflicting process |
|
|
| `No such process` | Stale PID file | Automatic cleanup on next hook trigger |
|
|
| `pm2: command not found` | PM2 not installed | None needed (error is caught and ignored) |
|
|
| `Invalid port X` | Port validation failed | Update `CLAUDE_MEM_WORKER_PORT` in settings |
|
|
|
|
## Developer Notes
|
|
|
|
### Testing the Migration
|
|
|
|
```bash
|
|
# 1. Install old version (with PM2)
|
|
git checkout <pre-7.1.0-tag>
|
|
npm install && npm run build && npm run sync-marketplace
|
|
|
|
# 2. Start PM2 worker
|
|
pm2 start plugin/scripts/worker-cli.js --name claude-mem-worker
|
|
|
|
# 3. Update to new version
|
|
git checkout main
|
|
npm install && npm run build && npm run sync-marketplace
|
|
|
|
# 4. Trigger hook
|
|
node plugin/scripts/session-start-hook.js
|
|
|
|
# 5. Verify migration
|
|
pm2 list # Should NOT show claude-mem-worker
|
|
cat ~/.claude-mem/.pm2-migrated # Should exist
|
|
npm run worker:status # Should show Bun worker running
|
|
```
|
|
|
|
### Architecture Decisions
|
|
|
|
**Why Custom ProcessManager Instead of PM2?**
|
|
1. **Simplicity**: Direct control, no external daemon
|
|
2. **Dependencies**: Remove npm dependency
|
|
3. **Cross-platform**: Bun handles platform differences
|
|
4. **Bundle Size**: Reduce plugin package size
|
|
5. **Control**: Fine-grained error handling and validation
|
|
|
|
**Why One-Time Marker Instead of Always Running PM2 Delete?**
|
|
1. **Performance**: Avoid unnecessary process spawning
|
|
2. **Idempotency**: Migration runs exactly once
|
|
3. **Debugging**: Timestamp shows when migration occurred
|
|
4. **Simplicity**: Clear migration state
|
|
|
|
**Why Run PM2 Cleanup on All Platforms?**
|
|
1. **Quality Migration**: Clean up orphaned processes
|
|
2. **Consistency**: Same behavior across all platforms
|
|
3. **Safety**: Error handling already in place (try/catch)
|
|
4. **No Downside**: If PM2 not installed, error is caught and ignored
|
|
|
|
## Summary
|
|
|
|
The migration from PM2 to Bun-based ProcessManager is a **one-time, automatic, transparent** transition that:
|
|
|
|
1. **Removes external dependencies** (PM2, better-sqlite3)
|
|
2. **Simplifies architecture** (direct process control)
|
|
3. **Improves cross-platform support** (especially Windows)
|
|
4. **Preserves user data** (database, settings, logs unchanged)
|
|
5. **Requires no user action** (automatic on first hook trigger)
|
|
|
|
**Key Migration Moment**: First hook trigger after update to 7.1.0+
|
|
**Duration**: ~2-5 seconds (one-time delay)
|
|
**Impact**: Seamless transition, user-invisible
|
|
**Rollback**: Not needed (migration is forward-only, safe)
|
|
|
|
For most users, the migration will be completely transparent - they'll see no errors, no data loss, and experience improved reliability and simpler troubleshooting going forward.
|