airkjw/claude-mem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Implement Worker Service with session management and SDK integration

Browse Source 
- Added WorkerService to handle long-running HTTP service with session management.
- Implemented endpoints for initializing, observing, finalizing, checking status, and deleting sessions.
- Integrated with Claude SDK for processing observations and generating responses.
- Added port allocator utility to dynamically find available ports for the service.
- Configured TypeScript settings for the project.
This commit is contained in:
Alex Newman
2025-10-17 15:59:36 -04:00
parent d6462919cb
commit 372854948c
57 changed files with 7055 additions and 6649 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/BUILD.md
-128
View File
@@ -1,128 +0,0 @@
# Build & Publish Guide
This repository is now the primary source for `claude-mem`. All builds and publishes happen from this repo.
## Prerequisites
- [Bun](https://bun.sh) - Fast JavaScript runtime and bundler
- npm account with publish access to `claude-mem`
## Building
Build the project to create a bundled, minified executable:
```bash
npm run build
# or
node scripts/build.js
```
This will:
1. Bundle all TypeScript source files using Bun
2. Minify the output
3. Add shebang (`#!/usr/bin/env node`)
4. Set executable permissions
5. Output to `dist/claude-mem.min.js`
### Build Output
- **Entry point:** `src/bin/cli.ts`
- **Output:** `dist/claude-mem.min.js` (~350KB minified)
- **Target:** Node.js (via Bun's `--target=node`)
- **Externals:** `@anthropic-ai/claude-agent-sdk` (not bundled)
## Publishing
To publish a new version to npm:
```bash
npm run publish:npm
# or
node scripts/publish.js
```
The publish script will:
1. Check git status (warn if uncommitted changes)
2. Show current version and prompt for version bump type:
- `patch` - Bug fixes (1.0.X)
- `minor` - New features (1.X.0)
- `major` - Breaking changes (X.0.0)
- `custom` - Enter version manually
3. Update `package.json` with new version
4. Run build script
5. Run tests (if configured)
6. Create git commit and tag (`v{version}`)
7. Publish to npm
8. Push commit and tags to GitHub
### Manual Publishing
If you prefer to do it manually:
```bash
# 1. Update version in package.json
# 2. Build
npm run build
# 3. Commit and tag
git add package.json dist/
git commit -m "Release v3.9.17"
git tag v3.9.17
# 4. Publish
npm publish
# 5. Push
git push && git push --tags
```
## Development
Run the CLI directly from source without building:
```bash
npm run dev -- [command] [options]
# or
bun run src/bin/cli.ts [command] [options]
```
Example:
```bash
npm run dev -- status
npm run dev -- --version
```
## File Structure
```
claude-mem/
├── src/ # TypeScript source
│ ├── bin/cli.ts # CLI entry point
│ ├── commands/ # Command implementations
│ ├── hooks/ # Hook implementations
│ ├── sdk/ # Agent SDK worker
│ ├── services/ # SQLite and path services
│ ├── shared/ # Configuration and types
│ └── utils/ # Platform utilities
├── dist/ # Build output
│ └── claude-mem.min.js # Bundled executable
├── tests/ # Test files
│ ├── database-schema.test.ts
│ ├── sdk-prompts-parser.test.ts
│ ├── hooks-database-integration.test.ts
│ └── session-lifecycle.test.ts
├── docs/ # Documentation
│ ├── BUILD.md # This file
│ └── CHANGELOG.md # Release notes
├── scripts/ # Build automation
│ ├── build.js # Build script
│ └── publish.js # Publish script
└── package.json # Package configuration
```
## Notes
- The build process embeds the version from `package.json` at build time
- `prepublishOnly` script ensures build runs before npm publish
- Dependencies are bundled except for external packages
- The published package includes: `dist/`, `commands/`, `src/`, `docs/`
docs/CHANGELOG.md
-119
View File
@@ -1,119 +0,0 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [3.7.1] - 2025-09-17
### Added
- SQLite storage backend with session, memory, overview, and diagnostics management
- Mintlify documentation site with searchable interface and comprehensive guides
- Context7 MCP integration for documentation retrieval
### Changed
- Session-start overviews to display chronologically from oldest to newest
### Fixed
- Migration index parsing bug that prevented JSONL records from importing to SQLite
## [3.6.10] - 2025-09-16
### Added
- Claude Code statusline integration for real-time memory status
- MCP memory tools server providing compress, stats, search, and overview commands
- Concept documentation explaining memory compression and context loading
### Fixed
- Corrected integration architecture to use hooks instead of MCP SDK
## [3.6.9] - 2025-09-14
### Added
- Display current date and time at the top of session-start hook output for better temporal context
### Changed
- Enhanced session-start hook formatting with emoji icons and separator lines for improved readability
## [3.6.8] - 2025-09-14
### Fixed
- Fixed publish command failing when no version-related memories exist for changelog generation
## [3.6.6] - 2025-09-14
### Fixed
- Resolved compaction errors when processing large conversation histories by reducing chunk size limits to stay within Claude's context window
## [3.6.5] - 2025-09-14
### Changed
- Session groups now display in chronological order (most recent first)
### Fixed
- Improved CLI path detection for cross-platform compatibility
## [3.6.4] - 2025-09-13
### Changed
- Update save documentation to include allowed-tools and description metadata fields
### Removed
- Remove deprecated markdown to JSONL migration script
## [3.6.3] - 2025-09-11
### Changed
- Updated changelog generation prompts to use date strings in query text for temporal filtering
### Fixed
- Resolved changelog timestamp filtering by using semantic search instead of metadata queries, enabling proper date-based searches
- Corrected install.ts search instructions to remove misleading metadata filtering guidance that caused 'Error finding id' errors
## [3.6.2] - 2025-09-10
### Added
- Visual feedback to changelog command showing current version, next version, and number of overviews being processed
- Generate changelog for specific versions using `--generate` flag with npm publish time boundaries
- Introduce 'Who Wants To Be a Memoryonaire?' trivia game that generates personalized questions from your stored memories
- Add interactive terminal UI with lifelines (50:50, Phone-a-Friend, Audience Poll) and cross-platform audio support
- Implement permanent question caching with --regenerate flag for instant game loading
- Enable hybrid vector search to discover related memory chains during question generation
### Changed
- Changelog regeneration automatically removes old entries from JSONL file when using `--generate` or `--historical` flags
- Switch to direct JSONL file loading for instant memory access without API calls
- Optimize AI generation with faster 'sonnet' model for improved performance
- Reduce memory query limit from 100 to 50 to prevent token overflow
### Fixed
- Changelog command now uses npm publish timestamps exclusively for accurate version time ranges
- Resolved timestamp filtering issues with Chroma database by leveraging semantic search with embedded dates
- Resolve game hanging at startup due to confirmation loop
- Fix memory integration bypass that prevented questions from using actual stored memories
- Consolidate 500+ lines of duplicate code for better maintainability
## [3.6.1] - 2025-09-10
### Changed
- Refactored pre-compact hook to work independently without status line updates
### Removed
- Removed status line integration and ccstatusline configuration support
## [3.5.5] - 2025-09-10
### Changed
- Standardized GitHub release naming to lowercase 'claude-mem vX.X.X' format for consistent branding
docs/CODEMAP.md
-1502
View File
File diff suppressed because it is too large Load Diff
docs/PLUGIN_DEVELOPMENT.md
-308
View File
@@ -1,308 +0,0 @@
# Plugin Development Guide
This guide helps developers work with the claude-mem plugin structure during development.
## Quick Start
### 1. Build the Plugin
```bash
# Build both CLI and hooks
npm run build
# Or build separately
npm run build:cli # Just the CLI
npm run build:hooks # Just the hooks
```
### 2. Test Hooks Locally
Test individual hooks by piping JSON input:
```bash
# Test context hook (SessionStart)
printf '{"session_id":"test-123","cwd":"/Users/you/project","source":"startup"}' | \
bun scripts/hooks/context-hook.js
# Test new hook (UserPromptSubmit)
printf '{"session_id":"test-123","cwd":"/Users/you/project","prompt":"help me code"}' | \
bun scripts/hooks/new-hook.js
# Test save hook (PostToolUse)
printf '{"session_id":"test-123","cwd":"/Users/you/project","tool_name":"Read","tool_input":{},"tool_output":{}}' | \
bun scripts/hooks/save-hook.js
# Test summary hook (Stop)
printf '{"session_id":"test-123","cwd":"/Users/you/project"}' | \
bun scripts/hooks/summary-hook.js
# Test worker (requires valid session ID in database)
bun scripts/hooks/worker.js 999
```
### 3. Test Worker with Plugin Root
Verify the new-hook correctly detects plugin mode:
```bash
# Without CLAUDE_PLUGIN_ROOT (traditional mode)
printf '{"session_id":"test-new","cwd":"/path","prompt":"test"}' | \
bun scripts/hooks/new-hook.js
# With CLAUDE_PLUGIN_ROOT (plugin mode)
CLAUDE_PLUGIN_ROOT=$(pwd) printf '{"session_id":"test-plugin","cwd":"/path","prompt":"test"}' | \
bun scripts/hooks/new-hook.js
```
In plugin mode, the new-hook will attempt to spawn `bun ${CLAUDE_PLUGIN_ROOT}/scripts/hooks/worker.js`.
In traditional mode, it will attempt to spawn `claude-mem worker`.
### 4. Test With No Input
Each hook should handle missing input gracefully:
```bash
echo '' | bun scripts/hooks/context-hook.js
# Output: No input provided - this script is designed to run as a Claude Code SessionStart hook
```
## Local Plugin Testing
### Option 1: Dev Marketplace (Recommended)
Create a development marketplace to test your plugin:
```bash
# Create marketplace structure
mkdir -p ~/dev-marketplace/.claude-plugin
# Create marketplace manifest
cat > ~/dev-marketplace/.claude-plugin/marketplace.json << 'EOF'
{
"name": "dev-marketplace",
"owner": {
"name": "Developer"
},
"plugins": [
{
"name": "claude-mem",
"source": "./claude-mem-plugin",
"description": "Persistent memory system for Claude Code"
}
]
}
EOF
# Symlink your working directory
ln -s /path/to/your/claude-mem ~/dev-marketplace/claude-mem-plugin
```
Then in Claude Code:
```
/plugin marketplace add /absolute/path/to/dev-marketplace
/plugin install claude-mem@dev-marketplace
```
### Option 2: Direct Testing
Test the CLI commands directly:
```bash
# Build first
npm run build
# Test CLI commands
./dist/claude-mem.min.js --version
./dist/claude-mem.min.js status
./dist/claude-mem.min.js --help
```
## Development Workflow
### Making Changes to Hooks
1. **Edit TypeScript source** in `src/hooks/` or `src/bin/hooks/`
2. **Rebuild hooks**: `npm run build:hooks`
3. **Test locally**: Use echo piping method above
4. **Reinstall plugin** (if testing in Claude Code):
```
/plugin uninstall claude-mem@dev-marketplace
/plugin install claude-mem@dev-marketplace
```
### Making Changes to CLI
1. **Edit TypeScript source** in `src/`
2. **Rebuild CLI**: `npm run build:cli`
3. **Test directly**: `./dist/claude-mem.min.js [command]`
### Making Changes to Commands
1. **Edit markdown files** in `commands/`
2. **No rebuild needed** (commands are read directly)
3. **Reinstall plugin** to pick up changes:
```
/plugin uninstall claude-mem@dev-marketplace
/plugin install claude-mem@dev-marketplace
```
## Debugging
### Enable Verbose Logging
Set environment variables for more detailed output:
```bash
DEBUG=claude-mem:* bun scripts/hooks/context-hook.js
```
### Check Hook Output
Hooks write to stdout/stderr. Capture output:
```bash
echo '{"session_id":"test","cwd":"/path"}' | \
bun scripts/hooks/context-hook.js 2>&1 | tee hook-output.log
```
### Verify Plugin Root Variable
Test that `${CLAUDE_PLUGIN_ROOT}` resolves correctly:
```bash
# Manually set it for testing
export CLAUDE_PLUGIN_ROOT=/path/to/your/plugin
echo '{"session_id":"test","cwd":"/path"}' | \
bun ${CLAUDE_PLUGIN_ROOT}/scripts/hooks/context-hook.js
```
## Build System Details
### Hook Build Process
The `scripts/build-hooks.js` script:
1. Reads each entry point from `src/bin/hooks/`
2. Bundles with Bun build system
3. Minifies output
4. Adds shebang for direct execution
5. Sets executable permissions
6. Outputs to `scripts/hooks/`
### CLI Build Process
The `scripts/build.js` script:
1. Bundles main CLI from `src/bin/cli.ts`
2. Externalizes large dependencies
3. Minifies output
4. Adds shebang
5. Sets executable permissions
6. Outputs to `dist/claude-mem.min.js`
### Build Configuration
Both builds use similar Bun configuration:
- **Target**: `bun` runtime
- **Minify**: `true`
- **External**: `bun:sqlite` (native module)
- **Define**: `__DEFAULT_PACKAGE_VERSION__` from package.json
## Testing
### Run Tests
```bash
bun test tests/
```
### Test Database Operations
```bash
# Test hooks database
bun test tests/hooks-database-integration.test.ts
# Test session lifecycle
bun test tests/session-lifecycle.test.ts
```
## Publishing
### Pre-publish Checklist
- [ ] All tests pass: `bun test tests/`
- [ ] Build succeeds: `npm run build`
- [ ] Version updated in `package.json`
- [ ] Changelog updated in `docs/CHANGELOG.md`
- [ ] Plugin.json version matches package.json
- [ ] Hooks tested locally
- [ ] CLI tested locally
### Publish to npm
```bash
npm run publish:npm
```
This will:
1. Run `prepublishOnly` script (builds everything)
2. Publish to npm registry
3. Include files listed in `package.json` "files" array
### Files Included in Package
The npm package includes:
- `dist/` - Compiled CLI
- `scripts/` - Compiled hooks
- `commands/` - Slash command definitions
- `hooks/` - Hook configuration
- `.claude-plugin/` - Plugin metadata
- `src/` - TypeScript source (for reference)
- `docs/` - Documentation
- `.mcp.json` - MCP server configuration
## Troubleshooting
### Build Fails
**Problem**: `bun: command not found`
**Solution**: Install Bun from https://bun.sh
**Problem**: Build errors with external dependencies
**Solution**: Check that `bun:sqlite` is not bundled (should be external)
### Hooks Don't Execute
**Problem**: `Permission denied` when executing hooks
**Solution**: Ensure scripts are executable: `chmod +x scripts/hooks/*.js`
**Problem**: Hooks exit silently
**Solution**: Check error handling - hooks catch all errors and exit gracefully
### Plugin Not Found
**Problem**: `/plugin install` can't find claude-mem
**Solution**:
1. Verify marketplace is added: `/plugin marketplace list`
2. Check marketplace manifest includes claude-mem
3. Refresh marketplace: `/plugin marketplace refresh`
## Tips
1. **Use symlinks** in dev marketplace for faster iteration
2. **Test hooks with edge cases** (empty input, malformed JSON)
3. **Check file sizes** after build to catch bloat
4. **Version everything together** (CLI, hooks, plugin.json)
5. **Document breaking changes** in CHANGELOG.md
## Resources
- [Plugin Structure Documentation](./PLUGIN_STRUCTURE.md)
- [Plugin Installation Guide](./PLUGIN_INSTALLATION.md)
- [Build Documentation](./BUILD.md)
- [Claude Code Plugins Docs](https://docs.claude.com/en/docs/claude-code/plugins)
- [Bun Documentation](https://bun.sh/docs)
## Getting Help
- **Issues**: https://github.com/thedotmack/claude-mem/issues
- **Discussions**: https://github.com/thedotmack/claude-mem/discussions
docs/PLUGIN_INSTALLATION.md
-273
View File
@@ -1,273 +0,0 @@
# Plugin Installation Guide
Claude-mem can be installed as a Claude Code plugin, which provides a streamlined installation experience.
## Plugin Structure
The claude-mem plugin includes:
- **Hooks**: Automatic memory capture via SessionStart, UserPromptSubmit, PostToolUse, and Stop hooks
- **Commands**: `/claude-mem`, `/save`, and `/remember` slash commands
- **MCP Integration**: Chroma vector database for semantic memory search
## Installation Methods
### Option 1: From Git Repository (Recommended)
If the plugin is published to GitHub:
1. **Add the marketplace from GitHub**:
```
/plugin marketplace add thedotmack/claude-mem-marketplace
```
2. **Install the plugin**:
```
/plugin install claude-mem@marketplace-name
```
The plugin will automatically:
- Configure all hooks in your Claude settings
- Set up slash commands
- Register plugin components
**Note**: You still need to install the CLI globally for hooks to work:
```bash
npm install -g claude-mem
```
### Option 2: Local Development Installation
If you're testing locally during development:
1. **Create a development marketplace structure**:
```bash
mkdir dev-marketplace
cd dev-marketplace
mkdir .claude-plugin
```
2. **Create marketplace manifest** (`.claude-plugin/marketplace.json`):
```json
{
"name": "dev-marketplace",
"owner": {
"name": "Developer"
},
"plugins": [
{
"name": "claude-mem",
"source": "./claude-mem-plugin",
"description": "Persistent memory system for Claude Code"
}
]
}
```
3. **Create or symlink your plugin directory**:
```bash
# Symlink to your working directory (recommended)
ln -s /path/to/claude-mem ./claude-mem-plugin
# Or copy the directory
cp -r /path/to/claude-mem ./claude-mem-plugin
```
4. **Start Claude Code and add your development marketplace**:
```
/plugin marketplace add /absolute/path/to/dev-marketplace
```
5. **Install your plugin**:
```
/plugin install claude-mem@dev-marketplace
```
### Option 3: Interactive Installation
You can also use the interactive plugin interface:
1. **Open plugin management**:
```
/plugin
```
2. **Select "Browse Plugins"** to see available plugins with descriptions and installation options
3. **Follow the prompts** to install and configure
### Option 4: Traditional CLI Installation
You can still use the traditional CLI-based installation:
```bash
npm install -g claude-mem
claude-mem install
```
This method uses an interactive wizard to guide you through setup and works independently of the plugin system.
## Verify Installation
After installing the plugin, verify it's working correctly:
1. **Check available commands**:
```
/help
```
You should see `/claude-mem`, `/save`, and `/remember` commands listed.
2. **Test plugin features**:
```
/claude-mem help
```
This should show all memory system commands and features.
3. **Review plugin details**:
```
/plugin
```
Select "Manage Plugins" to see what the plugin provides and its current status.
4. **Check hooks are registered**:
View your `~/.claude/settings.json` to confirm hooks are configured.
## Managing the Plugin
### Check Plugin Status
```
/plugin list
```
### Disable Plugin (temporarily)
```
/plugin disable claude-mem
```
### Enable Plugin
```
/plugin enable claude-mem
```
### Uninstall Plugin
```
/plugin uninstall claude-mem
```
## Development Iteration
When making changes to your plugin during development:
1. **Uninstall current version**:
```
/plugin uninstall claude-mem@dev-marketplace
```
2. **Make your changes** to the plugin code
3. **Reinstall to test changes**:
```
/plugin install claude-mem@dev-marketplace
```
4. **Restart Claude Code** if hooks or commands don't update immediately
**Tip**: When using a symlink in your dev marketplace, changes to commands may be reflected without reinstalling, but hook changes typically require a reinstall.
## Plugin Components
### Hooks Configuration
The plugin registers these hooks automatically (defined in `hooks/hooks.json`):
- **SessionStart**: Loads recent session context when Claude Code starts
- **UserPromptSubmit**: Initializes memory session and background worker
- **PostToolUse**: Captures tool observations for memory system
- **Stop**: Finalizes and saves session summary
### Commands
The plugin includes these slash commands (in `commands/` directory):
- `/claude-mem help` - Show all memory commands and features
- `/save [message]` - Quick save of current conversation overview
- `/remember [context]` - Search your saved memories
### MCP Server
The plugin automatically installs and configures the Chroma MCP server for vector-based memory storage.
## Troubleshooting
### Plugin Not Found
If the plugin isn't found, ensure:
1. The repository contains `.claude-plugin/plugin.json`
2. The plugin is in the correct directory
3. You've refreshed the plugin marketplace: `/plugin marketplace refresh`
### Hooks Not Running
Check that:
1. The plugin is enabled: `/plugin list`
2. The `claude-mem` CLI is installed: `which claude-mem`
3. Check Claude logs for hook errors
### CLI Not Found
If hooks fail with "command not found", install the CLI:
```bash
npm install -g claude-mem
```
The plugin hooks call the `claude-mem` CLI commands, which must be globally available.
## Creating Your Own Marketplace
To distribute claude-mem via a custom marketplace:
1. **Create a marketplace repository** with this structure:
```
my-marketplace/
├── .claude-plugin/
│ └── marketplace.json
└── plugins/
└── claude-mem/ (symlink or submodule to claude-mem)
```
2. **Define `.claude-plugin/marketplace.json`**:
```json
{
"name": "thedotmack",
"owner": {
"name": "Alex Newman"
},
"plugins": [
{
"name": "claude-mem",
"source": "./plugins/claude-mem",
"description": "Persistent memory system for Claude Code"
}
]
}
```
3. **Publish to GitHub** and users can add it with:
```
/plugin marketplace add username/repo-name
```
4. **Users install the plugin**:
```
/plugin install claude-mem@marketplace-name
```
## Benefits of Plugin Installation
✅ **One-command install**: No complex setup scripts
✅ **Easy management**: Enable/disable without modifying settings
✅ **Version control**: Update plugins with `/plugin update`
✅ **Team sharing**: Distribute via marketplace
✅ **Standard format**: Follows Claude Code best practices
## More Information
- [Claude Code Plugins Documentation](https://docs.claude.com/en/docs/claude-code/plugins)
- [Plugin Marketplaces Guide](https://docs.claude.com/en/docs/claude-code/plugin-marketplaces)
- [Claude-mem GitHub Repository](https://github.com/thedotmack/claude-mem)
docs/PLUGIN_STRUCTURE.md
-242
View File
@@ -1,242 +0,0 @@
# Claude-mem Plugin Structure
This document describes the complete plugin structure for claude-mem, which enables self-contained installation via Claude Code's plugin system.
## Directory Structure
```
claude-mem/
├── .claude-plugin/
│ └── plugin.json # Plugin metadata
├── commands/
│ ├── claude-mem.md # /claude-mem slash command
│ ├── remember.md # /remember slash command
│ └── save.md # /save slash command
├── hooks/
│ └── hooks.json # Hook definitions using ${CLAUDE_PLUGIN_ROOT}
├── scripts/
│ ├── build-hooks.js # Build script for compiling hooks
│ └── hooks/ # Compiled hook executables
│ ├── context-hook.js # SessionStart hook (4KB)
│ ├── new-hook.js # UserPromptSubmit hook (4KB)
│ ├── save-hook.js # PostToolUse hook (4KB)
│ ├── summary-hook.js # Stop hook (4KB)
│ └── worker.js # Background SDK worker (232KB)
├── src/
│ ├── bin/
│ │ ├── cli.ts # Main CLI entry point
│ │ └── hooks/ # Hook entry point sources
│ │ ├── context-hook.ts # SessionStart entry point
│ │ ├── new-hook.ts # UserPromptSubmit entry point
│ │ ├── save-hook.ts # PostToolUse entry point
│ │ └── summary-hook.ts # Stop entry point
│ ├── hooks/ # Hook implementation functions
│ │ ├── context.ts
│ │ ├── new.ts
│ │ ├── save.ts
│ │ └── summary.ts
│ └── ... # Other source files
└── dist/
└── claude-mem.min.js # Bundled CLI executable
```
## How It Works
### 1. Plugin Installation
When users install the plugin via `/plugin install claude-mem`, Claude Code:
1. Downloads the plugin from the marketplace
2. Installs it to the local plugins directory
3. Registers the hooks from `hooks/hooks.json`
4. Makes slash commands from `commands/` directory available
### 2. Self-Contained Execution
The hooks are compiled as standalone executables that:
- **Don't require global CLI installation**: All dependencies are bundled
- **Use plugin-relative paths**: `${CLAUDE_PLUGIN_ROOT}` resolves to plugin directory
- **Work with Bun runtime**: Scripts are compiled for Bun and include shebang
### 3. Hook Configuration
The `hooks/hooks.json` file uses `${CLAUDE_PLUGIN_ROOT}` to reference bundled scripts:
```json
{
"description": "Claude-mem memory system hooks",
"hooks": {
"SessionStart": [{
"hooks": [{
"type": "command",
"command": "bun ${CLAUDE_PLUGIN_ROOT}/scripts/hooks/context-hook.js",
"timeout": 180000
}]
}],
...
}
}
```
### 4. Hook Entry Points
Each hook has a standalone entry point in `src/bin/hooks/` that:
- Reads JSON input from stdin
- Calls the hook implementation function
- Handles errors gracefully
- Exits with appropriate status codes
Example from `context-hook.ts`:
```typescript
#!/usr/bin/env bun
import { contextHook } from '../../hooks/context.js';
const input = await Bun.stdin.text();
const parsed = input.trim() ? JSON.parse(input) : undefined;
contextHook(parsed);
```
### 5. Build Process
The build system compiles both the CLI and hook scripts:
```bash
npm run build # Build both CLI and hooks
npm run build:cli # Build only the CLI
npm run build:hooks # Build only the hooks
```
The hook build process:
1. Compiles each hook entry point with Bun
2. Bundles all dependencies (except bun:sqlite)
3. Minifies the output
4. Adds shebang (`#!/usr/bin/env bun`)
5. Makes executable (`chmod +x`)
6. Outputs to `scripts/hooks/`
## Benefits
### ✅ Self-Contained
- No global CLI installation required
- All dependencies bundled with plugin
- Plugin directory has everything needed
### ✅ Easy Installation
- Single command: `/plugin install claude-mem`
- Hooks automatically registered
- Slash commands immediately available
### ✅ Version Control
- Plugin version tied to specific hook versions
- No version mismatch between CLI and hooks
- Easy updates via `/plugin update`
### ✅ Development Friendly
- Source code in TypeScript
- Compiled to optimized JavaScript
- Fast execution with Bun runtime
## Usage
### For Users
Install the plugin:
```
/plugin install claude-mem@marketplace-name
```
The plugin provides:
- **Hooks**: Automatic memory capture on SessionStart, UserPromptSubmit, PostToolUse, Stop
- **Commands**: `/claude-mem`, `/save`, `/remember` slash commands
- **MCP Integration**: Chroma vector database access via MCP tools
### For Developers
Build the plugin:
```bash
npm run build
```
Test hooks locally:
```bash
echo '{"session_id":"test","cwd":"/path"}' | bun scripts/hooks/context-hook.js
```
Publish to marketplace:
```bash
npm run publish:npm
```
## Worker Process Handling
### Background Worker
The `worker.js` is a special bundled script that runs as a long-lived background process. It:
- Runs an SDK agent session in the background
- Listens on a Unix socket for messages from hooks
- Processes tool observations and generates summaries
- Stores results in the database
### Spawning the Worker
The `new-hook` (UserPromptSubmit) is responsible for spawning the worker. It uses intelligent fallback:
```typescript
// Plugin mode: Use bundled worker with CLAUDE_PLUGIN_ROOT
if (process.env.CLAUDE_PLUGIN_ROOT) {
const workerPath = path.join(pluginRoot, 'scripts', 'hooks', 'worker.js');
spawn('bun', [workerPath, sessionId.toString()], { detached: true });
}
// Traditional mode: Use global CLI
else {
spawn('claude-mem', ['worker', sessionId.toString()], { detached: true });
}
```
### Why This Approach?
1. **Self-contained plugin**: When installed as a plugin, uses bundled worker
2. **Backwards compatible**: When installed traditionally, uses global CLI
3. **No user intervention**: Automatically detects mode via environment variable
## File Sizes
Compiled hook scripts are optimized and small:
- `context-hook.js`: ~4.1 KB
- `new-hook.js`: ~4.0 KB
- `save-hook.js`: ~4.2 KB
- `summary-hook.js`: ~3.9 KB
- `worker.js`: ~232 KB (includes SDK dependencies)
Total package overhead: ~248 KB for all hook scripts combined.
## Dependencies
### Runtime
- **Bun**: Required for executing hook scripts
- **bun:sqlite**: Native SQLite module (not bundled)
### Build-time
- **Bun**: Used as bundler for compilation
- **Node.js**: Required for build scripts
## Backwards Compatibility
The plugin structure maintains backwards compatibility:
- CLI commands still work: `claude-mem context`, etc.
- Traditional installation still supported: `npm install -g claude-mem`
- Users can choose plugin OR CLI installation
## Future Enhancements
Potential improvements:
- [ ] Add more hooks (e.g., PreToolUse, Error)
- [ ] Support Node.js runtime in addition to Bun
- [ ] Add hook configuration UI
- [ ] Implement hook hot-reloading during development
- [ ] Create plugin marketplace for distribution
## See Also
- [Plugin Installation Guide](./PLUGIN_INSTALLATION.md) - User-facing installation instructions
- [Build Documentation](./BUILD.md) - Build system details
- [Claude Code Plugins Docs](https://docs.claude.com/en/docs/claude-code/plugins) - Official plugin documentation
docs/WORKER_ARCHITECTURE.md
-272
View File
@@ -1,272 +0,0 @@
# Worker Process Architecture
This document explains how the SDK worker process is handled in both plugin and traditional installation modes.
## Overview
The worker is a critical background process that:
- Runs the Claude Agent SDK in a long-lived session
- Listens on a Unix socket for messages from hooks
- Processes tool observations in real-time
- Generates session summaries
- Stores observations and summaries in the database
## Architecture Diagram
```
┌─────────────────┐
│ Claude Code │
│ (Main UI) │
└────────┬────────┘
│
├─ SessionStart Hook ──> context-hook.js
│
├─ UserPromptSubmit ───> new-hook.js ──┐
│ │
│ ├──> Spawns Worker
│ │ (Background Process)
│ │
├─ PostToolUse Hook ───> save-hook.js ─┼──> Unix Socket
│ │
│ │ ┌──────────────┐
│ └───>│ worker.js │
│ │ │
│ │ - SDK Agent │
└─ Stop Hook ───────────> summary-hook.js ──┼─>- Socket Srv│
│ - Database │
└──────────────┘
```
## Worker Lifecycle
### 1. Session Initialization (UserPromptSubmit)
When a user submits a prompt, `new-hook.js` is triggered:
```typescript
// 1. Check if session already exists
const existing = db.findActiveSDKSession(session_id);
if (existing) { /* already running */ }
// 2. Create new SDK session record
const sessionId = db.createSDKSession(session_id, project, prompt);
// 3. Spawn worker process
const pluginRoot = process.env.CLAUDE_PLUGIN_ROOT;
if (pluginRoot) {
// Plugin mode: use bundled worker
spawn('bun', [`${pluginRoot}/scripts/hooks/worker.js`, sessionId]);
} else {
// Traditional mode: use global CLI
spawn('claude-mem', ['worker', sessionId]);
}
```
### 2. Worker Startup
The worker process starts and:
1. Loads session info from database
2. Creates Unix socket at `~/.claude-mem/sockets/session-{id}.sock`
3. Starts listening for messages
4. Initializes SDK agent with streaming input
### 3. Tool Observation Flow (PostToolUse)
Every time Claude uses a tool, `save-hook.js` is triggered:
```typescript
// 1. Find active SDK session
const session = db.findActiveSDKSession(session_id);
// 2. Connect to worker's Unix socket
const socketPath = getWorkerSocketPath(session.id);
const client = net.connect(socketPath);
// 3. Send observation message
client.write(JSON.stringify({
type: 'observation',
tool_name: 'Read',
tool_input: '{"file_path": "/path/to/file"}',
tool_output: '{"content": "..."}'
}));
```
The worker receives the message and:
1. Queues it in `pendingMessages`
2. Yields it to SDK agent via message generator
3. Receives agent's analysis
4. Parses and stores observations in database
### 4. Session Finalization (Stop)
When the session ends, `summary-hook.js` is triggered:
```typescript
// 1. Find active session
const session = db.findActiveSDKSession(session_id);
// 2. Send finalize message to worker
const client = net.connect(socketPath);
client.write(JSON.stringify({
type: 'finalize'
}));
```
The worker:
1. Stops accepting new observations
2. Sends finalize prompt to SDK agent
3. Receives and parses session summary
4. Stores summary in database
5. Cleans up socket and exits
## Plugin vs Traditional Mode
### Plugin Mode (Self-Contained)
When installed as a plugin:
- `CLAUDE_PLUGIN_ROOT` environment variable is set by Claude Code
- Hooks use bundled scripts: `${CLAUDE_PLUGIN_ROOT}/scripts/hooks/`
- Worker is spawned as: `bun ${CLAUDE_PLUGIN_ROOT}/scripts/hooks/worker.js`
- **No global CLI installation required**
```bash
# Plugin mode execution
/plugin install claude-mem
# Hooks automatically use bundled worker
```
### Traditional Mode (Global CLI)
When installed via npm:
- `CLAUDE_PLUGIN_ROOT` is not set
- Hooks installed via `claude-mem install`
- Worker is spawned as: `claude-mem worker`
- **Requires global CLI installation**
```bash
# Traditional mode installation
npm install -g claude-mem
claude-mem install
```
## Worker Binary Size
The worker is the largest bundled script because it includes:
- Claude Agent SDK (`@anthropic-ai/claude-agent-sdk`)
- Prompt templates and parsers
- Socket server implementation
- Database operations
**Size**: ~232 KB (minified)
This is acceptable because:
- Only spawned once per session (not per tool use)
- Runs in background (doesn't block UI)
- Contains full SDK functionality
## Worker Communication Protocol
### Message Types
#### 1. Observation Message
```json
{
"type": "observation",
"tool_name": "Read",
"tool_input": "{\"file_path\": \"/path\"}",
"tool_output": "{\"content\": \"...\"}"
}
```
#### 2. Finalize Message
```json
{
"type": "finalize"
}
```
### Socket Protocol
- **Transport**: Unix domain socket
- **Format**: JSON messages separated by newlines (`\n`)
- **Location**: `~/.claude-mem/sockets/session-{id}.sock`
- **Lifecycle**: Created on worker startup, deleted on worker exit
## Error Handling
### Worker Startup Failures
If the worker fails to start:
- New-hook logs error but doesn't block Claude Code
- Session record remains in "pending" state
- No observations are captured (graceful degradation)
### Socket Communication Failures
If hooks can't connect to worker socket:
- Hook logs error but doesn't block Claude Code
- Tool use continues normally
- Observations are skipped for that session
### Worker Crashes
If the worker crashes mid-session:
- Database marks session as "failed"
- Socket is cleaned up automatically
- Next session will spawn new worker
## Testing
### Test Worker Directly
```bash
# Build the worker
npm run build:hooks
# Test worker (needs valid session ID in DB)
bun scripts/hooks/worker.js 123
```
### Test Worker Spawning
```bash
# Plugin mode (with CLAUDE_PLUGIN_ROOT)
CLAUDE_PLUGIN_ROOT=$(pwd) printf '{"session_id":"test","cwd":"/path","prompt":"help"}' | \
bun scripts/hooks/new-hook.js
# Traditional mode (without CLAUDE_PLUGIN_ROOT)
printf '{"session_id":"test","cwd":"/path","prompt":"help"}' | \
bun scripts/hooks/new-hook.js
```
### Monitor Worker Logs
Worker logs to stderr:
```bash
# Watch worker logs in real-time
tail -f ~/.claude-mem/logs/worker-*.log
```
## Benefits of This Architecture
1. **Self-contained**: Plugin bundles everything needed
2. **Backwards compatible**: Works with global CLI too
3. **Automatic detection**: Uses environment variable to choose mode
4. **Isolated execution**: Worker runs in separate process
5. **Async communication**: Hooks don't block on SDK operations
6. **Graceful degradation**: Failures don't break Claude Code
## Future Improvements
Potential enhancements:
- [ ] Worker health checks and auto-restart
- [ ] Multiple workers for concurrent sessions
- [ ] Worker pool management
- [ ] WebSocket support for remote workers
- [ ] Worker performance metrics
## See Also
- [Plugin Structure Documentation](./PLUGIN_STRUCTURE.md)
- [Plugin Development Guide](./PLUGIN_DEVELOPMENT.md)
- [Build Documentation](./BUILD.md)
docs/plans/phase0-task1-summary.md
-165
View File
@@ -1,165 +0,0 @@
# Phase 0 Task 1: Add Comprehensive Logging to Summary Hook
## Overview
Added comprehensive logging to the Stop hook (summary hook) to verify it fires on normal exit and successfully sends the FINALIZE message to the worker socket.
## Files Modified
### `/Users/alexnewman/Scripts/claude-mem/src/hooks/summary.ts`
Added 8 logging points throughout the hook execution flow.
## Logging Points Added
All logs use the `[claude-mem summary]` prefix for easy searching and use `console.error()` to output to stderr (visible in terminal).
### 1. Hook Entry Point (Line 18-20)
```typescript
console.error('[claude-mem summary] Hook fired', {
input: input ? { session_id: input.session_id, cwd: input.cwd } : null
});
```
**Purpose:** Confirms the hook was called by Claude Code and logs the input parameters.
### 2. Session Search (Line 34)
```typescript
console.error('[claude-mem summary] Searching for active SDK session', { session_id });
```
**Purpose:** Logs the session_id being searched for in the database.
### 3. Session Not Found (Line 43)
```typescript
console.error('[claude-mem summary] No active SDK session found', { session_id });
```
**Purpose:** Logs when no active session is found (normal for non-SDK sessions).
### 4. Session Found (Line 48-52)
```typescript
console.error('[claude-mem summary] Active SDK session found', {
session_id: session.id,
collection_name: session.collection_name,
worker_pid: session.worker_pid
});
```
**Purpose:** Logs when an active session is found with its details for verification.
### 5. Before Socket Send (Line 62-65)
```typescript
console.error('[claude-mem summary] Attempting to send FINALIZE message to worker socket', {
socketPath,
message
});
```
**Purpose:** Logs the socket path and message content before attempting connection.
### 6. Socket Connection Established (Line 68)
```typescript
console.error('[claude-mem summary] Socket connection established, sending message');
```
**Purpose:** Confirms successful socket connection before writing data.
### 7. Socket Error Handler (Line 75-79)
```typescript
console.error('[claude-mem summary] Socket error occurred', {
error: err.message,
code: (err as any).code,
socketPath
});
```
**Purpose:** Logs detailed error information if socket connection fails (includes error code like ENOENT, ECONNREFUSED).
### 8. Socket Close Handler (Line 84)
```typescript
console.error('[claude-mem summary] Socket connection closed successfully');
```
**Purpose:** Confirms the socket connection closed cleanly after sending message.
### 9. Catch Block (Line 91-95)
```typescript
console.error('[claude-mem summary] Unexpected error in hook', {
error: error.message,
stack: error.stack,
name: error.name
});
```
**Purpose:** Logs any unexpected errors with full stack trace for debugging.
## How to Test
### Basic Test (Normal Exit)
1. Start a Claude Code session in a project with claude-mem configured
2. Have a conversation that triggers SDK memory operations
3. Exit Claude Code normally (Ctrl+D or type "exit")
4. Check terminal stderr for log sequence:
```
[claude-mem summary] Hook fired
[claude-mem summary] Searching for active SDK session
[claude-mem summary] Active SDK session found
[claude-mem summary] Attempting to send FINALIZE message to worker socket
[claude-mem summary] Socket connection established, sending message
[claude-mem summary] Socket connection closed successfully
```
### Test Cases
#### Case 1: Normal Exit with Active Session
**Expected logs:**
1. Hook fired (with session_id and cwd)
2. Searching for active SDK session
3. Active SDK session found (with session details)
4. Attempting to send FINALIZE message (with socket path)
5. Socket connection established
6. Socket connection closed successfully
#### Case 2: Exit with No Active Session
**Expected logs:**
1. Hook fired
2. Searching for active SDK session
3. No active SDK session found
#### Case 3: Worker Socket Already Closed
**Expected logs:**
1. Hook fired
2. Searching for active SDK session
3. Active SDK session found
4. Attempting to send FINALIZE message
5. Socket error occurred (with ENOENT or ECONNREFUSED code)
#### Case 4: Database Error
**Expected logs:**
1. Hook fired
2. Searching for active SDK session
3. Unexpected error in hook (with stack trace)
### Log Filtering
To view only summary hook logs:
```bash
claude-code 2>&1 | grep "\[claude-mem summary\]"
```
## Behavior Guarantees
1. **No Breaking Changes:** All existing functionality remains identical
2. **Non-Blocking:** All errors are caught and logged but don't block Claude Code
3. **Clean Exit:** Hook always returns proper JSON response to Claude Code
4. **Searchable:** All logs use consistent `[claude-mem summary]` prefix
## Issues and Concerns
### None Discovered
- The existing error handling is robust
- All error paths properly log and exit gracefully
- No changes needed to logic, only observability added
### Potential Observations During Testing
- If socket errors are common, may indicate worker timing issues
- If "No active SDK session found" appears frequently, may indicate database query issues
- If hook never fires, indicates Claude Code hook registration problem
- If socket path is wrong, indicates paths.ts configuration issue
## Next Steps
After testing with these logs:
1. Verify hook fires on every Claude Code exit
2. Verify FINALIZE message reaches worker socket
3. Check for any unexpected error patterns
4. Use logs to diagnose any issues with worker finalization flow
docs/plans/phase0-task2-summary.md
-323
View File
@@ -1,323 +0,0 @@
# Phase 0 Task 2: SDK Worker Comprehensive Logging
## Summary
Added comprehensive logging to `/Users/alexnewman/Scripts/claude-mem/src/sdk/worker.ts` to trace the complete flow of the FINALIZE message from receipt through SDK agent processing to database storage.
## Modified Files
1. `/Users/alexnewman/Scripts/claude-mem/src/sdk/worker.ts` - Added 20+ logging points throughout the worker lifecycle
## Logging Points Added
All logs use the `[claude-mem worker]` prefix for easy searching and are sent to stderr using `console.error()`.
### 1. Worker Initialization (Lines 70-73)
- **Location:** `constructor()`
- **What:** Logs when worker instance is created
- **Data:** sessionDbId, socketPath
### 2. Worker Run Started (Lines 80-83)
- **Location:** `run()` method entry
- **What:** Logs when main run loop begins
- **Data:** sessionDbId, socketPath
### 3. Session Loading (Lines 89-100)
- **Location:** `run()` method after `loadSession()`
- **What:** Logs session load failure or success
- **Data:**
- Failure: sessionDbId
- Success: sessionDbId, project, sdkSessionId, userPromptLength
### 4. Socket Server Started (Lines 107-110)
- **Location:** `run()` method after `startSocketServer()`
- **What:** Logs successful socket server initialization
- **Data:** socketPath, sessionDbId
### 5. SDK Agent Starting (Lines 113-116)
- **Location:** `run()` method before `runSDKAgent()`
- **What:** Logs SDK agent invocation
- **Data:** sessionDbId, model
### 6. SDK Agent Completed (Lines 120-123)
- **Location:** `run()` method after `runSDKAgent()` completes
- **What:** Logs completion before marking session as done
- **Data:** sessionDbId, sdkSessionId
### 7. Fatal Error Handler (Lines 129-133)
- **Location:** `run()` method catch block
- **What:** Logs any fatal errors with full stack trace
- **Data:** sessionDbId, error message, stack trace
### 8. Socket Connection Received (Lines 157-160)
- **Location:** `startSocketServer()` - connection handler
- **What:** Logs when a client connects to the Unix socket
- **Data:** sessionDbId, socketPath
### 9. Data Received on Socket (Lines 164-167)
- **Location:** `startSocketServer()` - data handler
- **What:** Logs when data arrives on socket
- **Data:** sessionDbId, chunk size
### 10. Message Parsed from Socket (Lines 178-182)
- **Location:** `startSocketServer()` - message parsing
- **What:** Logs successfully parsed JSON message
- **Data:** sessionDbId, messageType, rawMessage (truncated to 500 chars)
### 11. Invalid Message Error (Lines 185-189)
- **Location:** `startSocketServer()` - JSON parse error
- **What:** Logs when message fails to parse
- **Data:** sessionDbId, error message, rawLine (truncated to 200 chars)
### 12. Socket Connection Error (Lines 196-200)
- **Location:** `startSocketServer()` - socket error handler
- **What:** Logs socket-level errors
- **Data:** sessionDbId, error message, stack trace
### 13. Server Errors (Lines 206-216)
- **Location:** `startSocketServer()` - server error handler
- **What:** Logs server-level errors (EADDRINUSE, etc.)
- **Data:** sessionDbId, socketPath (if EADDRINUSE), error details
### 14. Message Handler Entry (Lines 233-237)
- **Location:** `handleMessage()` method entry
- **What:** Logs when processing any message
- **Data:** sessionDbId, messageType, pendingMessagesCount
### 15. FINALIZE Message Detected (Lines 242-246)
- **Location:** `handleMessage()` - finalize detection
- **What:** Logs when FINALIZE message is received (CRITICAL LOG)
- **Data:** sessionDbId, isFinalized=true, pendingMessagesCount
### 16. Observation Message Queued (Lines 249-254)
- **Location:** `handleMessage()` - observation handling
- **What:** Logs observation message details
- **Data:** sessionDbId, toolName, input/output lengths
### 17. SDK Session Initialized (Lines 292-295)
- **Location:** `runSDKAgent()` - onSystemInitMessage callback
- **What:** Logs when SDK session ID is received
- **Data:** sessionDbId, sdkSessionId
### 18. SDK Agent Response Received (Lines 301-306)
- **Location:** `runSDKAgent()` - onAgentMessage callback
- **What:** Logs every response from SDK agent (CRITICAL LOG)
- **Data:** sessionDbId, sdkSessionId, contentLength, contentPreview (200 chars)
### 19. Initial Prompt Yielded (Lines 322-327)
- **Location:** `createMessageGenerator()` - initial prompt
- **What:** Logs when first prompt is sent to SDK agent
- **Data:** sessionDbId, claudeSessionId, project, promptLength
### 20. FINALIZE Processing in Generator (Lines 349-352)
- **Location:** `createMessageGenerator()` - finalize handling
- **What:** Logs when FINALIZE is processed in async generator (CRITICAL LOG)
- **Data:** sessionDbId, sdkSessionId
### 21. Finalize Prompt Yielded (Lines 357-362)
- **Location:** `createMessageGenerator()` - after building finalize prompt
- **What:** Logs finalize prompt being sent to SDK agent (CRITICAL LOG)
- **Data:** sessionDbId, sdkSessionId, promptLength, promptPreview (300 chars)
### 22. Failed to Load Session for Finalize (Lines 371-373)
- **Location:** `createMessageGenerator()` - error case
- **What:** Logs if session reload fails during finalize
- **Data:** sessionDbId
### 23. Observation Prompt Yielded (Lines 385-389)
- **Location:** `createMessageGenerator()` - observation handling
- **What:** Logs when observation prompt is sent to SDK agent
- **Data:** sessionDbId, toolName, promptLength
### 24. Parsing Agent Message (Lines 406-410)
- **Location:** `handleAgentMessage()` method entry
- **What:** Logs when starting to parse agent response
- **Data:** sessionDbId, sdkSessionId, contentLength
### 25. Observations Parsed (Lines 414-418)
- **Location:** `handleAgentMessage()` - after parseObservations()
- **What:** Logs how many observations were found
- **Data:** sessionDbId, sdkSessionId, observationCount
### 26. Storing Observation (Lines 422-428)
- **Location:** `handleAgentMessage()` - in observation loop
- **What:** Logs each observation being stored
- **Data:** sessionDbId, sdkSessionId, project, observationType, observationTextLength
### 27. Cannot Store Observation (Lines 431-434)
- **Location:** `handleAgentMessage()` - error case
- **What:** Logs when SDK session ID is missing
- **Data:** sessionDbId, observationType
### 28. Attempting to Parse Summary (Lines 439-442)
- **Location:** `handleAgentMessage()` - before parseSummary()
- **What:** Logs when attempting summary parse (CRITICAL LOG)
- **Data:** sessionDbId, sdkSessionId
### 29. Summary Parsed Successfully (Lines 446-456)
- **Location:** `handleAgentMessage()` - after parseSummary() success
- **What:** Logs summary structure details (CRITICAL LOG)
- **Data:** sessionDbId, sdkSessionId, project, hasRequest, hasInvestigated, hasLearned, hasCompleted, filesReadCount, filesEditedCount
### 30. Storing Summary in Database (Lines 470-474)
- **Location:** `handleAgentMessage()` - before storeSummary()
- **What:** Logs summary about to be stored (CRITICAL LOG)
- **Data:** sessionDbId, sdkSessionId, project
### 31. Summary Stored Successfully (Lines 478-482)
- **Location:** `handleAgentMessage()` - after storeSummary()
- **What:** Logs successful database storage (CRITICAL LOG)
- **Data:** sessionDbId, sdkSessionId, project
### 32. Summary Parsed but No SDK Session (Lines 484-486)
- **Location:** `handleAgentMessage()` - error case
- **What:** Logs when summary found but can't store
- **Data:** sessionDbId
### 33. No Summary Found (Lines 488-491)
- **Location:** `handleAgentMessage()` - no summary case
- **What:** Logs when response has no summary
- **Data:** sessionDbId, sdkSessionId
### 34. Cleanup Started (Lines 499-504)
- **Location:** `cleanup()` method entry
- **What:** Logs cleanup process beginning
- **Data:** sessionDbId, socketPath, hasServer, socketExists
### 35. Cleanup Complete (Lines 513-515)
- **Location:** `cleanup()` method exit
- **What:** Logs cleanup finished
- **Data:** sessionDbId
## How to Test
### 1. Start the Worker
```bash
# Start a worker for session ID 1 (for example)
bun run src/sdk/worker.ts 1
```
Look for logs:
- `[claude-mem worker] Worker instance created`
- `[claude-mem worker] Worker run() started`
- `[claude-mem worker] Session loaded successfully`
- `[claude-mem worker] Socket server started successfully`
- `[claude-mem worker] Starting SDK agent`
### 2. Send Messages via Socket
```bash
# From another terminal, send a message to the socket
# Socket path format: /tmp/claude-mem-worker-{sessionDbId}.sock
# Send an observation
echo '{"type":"observation","tool_name":"Read","tool_input":"...","tool_output":"..."}' | nc -U /tmp/claude-mem-worker-1.sock
# Send finalize
echo '{"type":"finalize"}' | nc -U /tmp/claude-mem-worker-1.sock
```
### 3. Monitor Logs
Use grep to filter for specific events:
```bash
# All worker logs
bun run src/sdk/worker.ts 1 2>&1 | grep '\[claude-mem worker\]'
# Only FINALIZE-related logs
bun run src/sdk/worker.ts 1 2>&1 | grep -i finalize
# Only summary-related logs
bun run src/sdk/worker.ts 1 2>&1 | grep -i summary
# Only database storage logs
bun run src/sdk/worker.ts 1 2>&1 | grep -i storing
```
## What to Look for When FINALIZE is Sent
The expected log sequence when a FINALIZE message is sent:
1. **Message Receipt:**
```
[claude-mem worker] Data received on socket
[claude-mem worker] Message received from socket { messageType: 'finalize', ... }
```
2. **Message Handling:**
```
[claude-mem worker] Processing message in handleMessage() { messageType: 'finalize', ... }
[claude-mem worker] FINALIZE message detected { isFinalized: true, ... }
```
3. **Generator Processing:**
```
[claude-mem worker] Processing FINALIZE message in generator
[claude-mem worker] Yielding finalize prompt to SDK agent { promptLength: ..., promptPreview: ... }
```
4. **SDK Agent Response:**
```
[claude-mem worker] SDK agent response received { contentLength: ..., contentPreview: ... }
```
5. **Parsing and Storage:**
```
[claude-mem worker] Parsing agent message for observations and summary
[claude-mem worker] Observations parsed from response { observationCount: ... }
[claude-mem worker] Attempting to parse summary from response
[claude-mem worker] Summary parsed successfully { hasRequest: true, hasLearned: true, ... }
[claude-mem worker] Storing summary in database
[claude-mem worker] Summary stored successfully in database
```
6. **Completion:**
```
[claude-mem worker] SDK agent completed, marking session as completed
[claude-mem worker] Cleaning up worker resources
[claude-mem worker] Cleanup complete
```
## Issues and Concerns
### 1. Large Response Truncation
- Raw messages are truncated to 500 chars in socket logs
- Content previews are limited to 200-300 chars
- This prevents log spam but might make debugging harder if the critical info is beyond the truncation point
### 2. Async Generator Timing
- The generator waits in a loop (`while (!this.isFinalized)`) with 100ms sleeps
- Logs show when messages are queued but not when the generator processes them
- There could be a small delay between "FINALIZE message detected" and "Processing FINALIZE in generator"
### 3. Error Cases Not Fully Logged
- Parser errors in `parseObservations()` and `parseSummary()` are not logged
- Should consider adding try-catch in `handleAgentMessage()` to catch parser exceptions
- XML parsing errors would be silent
### 4. No Timing Information
- Logs don't include timestamps (relies on stderr default timestamps)
- Could add `Date.now()` or elapsed time to measure performance bottlenecks
### 5. Socket Path Permissions
- No logging for socket file permissions or creation errors
- If socket can't be created due to permissions, error might not be clear
### 6. Multi-Message Batching
- If multiple messages arrive rapidly, they're processed in a batch
- Logs show individual messages but don't indicate batch boundaries
- Could add batch ID or sequence numbers
## Recommendations for Next Steps
1. **Test the logging** by running the worker and sending various messages
2. **Add parser error handling** in `handleAgentMessage()` to catch XML parse failures
3. **Consider adding timing metrics** to measure latency at each stage
4. **Validate socket connectivity** early in startup (try writing a test message)
5. **Add structured logging library** if JSON logs would be easier to parse programmatically
## Related Files
- `/Users/alexnewman/Scripts/claude-mem/src/sdk/prompts.ts` - Prompt builders used in logged operations
- `/Users/alexnewman/Scripts/claude-mem/src/sdk/parser.ts` - XML parsers for observations and summaries
- `/Users/alexnewman/Scripts/claude-mem/src/services/sqlite/HooksDatabase.js` - Database methods being called
- `/Users/alexnewman/Scripts/claude-mem/src/shared/paths.js` - Socket path generation
docs/plans/phase0-task2b-fixes.md
-100
View File
@@ -1,100 +0,0 @@
# Phase 0 Task 2b: TypeScript Error Fixes
## Summary
Fixed all 6 TypeScript errors in `src/sdk/worker.ts` that were introduced after adding logging functionality. All logging has been preserved.
## Errors Fixed
### 1. Line 283 (now 338): Type error with AsyncIterable - missing `parent_tool_use_id` property
**Error**: Return type was `AsyncIterable<{ type: 'user'; message: { role: 'user'; content: string } }>` which didn't match the SDK's `SDKUserMessage` type.
**Fix**:
- Changed return type to `AsyncIterable<SDKUserMessage>`
- Added required `session_id` and `parent_tool_use_id: null` properties to all yielded messages
- Updated all three yield statements in the generator (initial prompt, finalize prompt, and observation prompt)
**Changes**:
- Line 338: Updated function signature
- Lines 348-356: Added `session_id` and `parent_tool_use_id` to initial prompt yield
- Lines 385-393: Added `session_id` and `parent_tool_use_id` to finalize prompt yield
- Lines 416-424: Added `session_id` and `parent_tool_use_id` to observation prompt yield
### 2. Line 289 (now removed): `onSystemInitMessage` doesn't exist in type 'Options'
**Error**: The `Options` type from the Claude Agent SDK doesn't have an `onSystemInitMessage` callback property.
**Fix**:
- Removed the invalid callback options from the `query()` call
- Changed to iterate over the returned `Query` async generator
- Handle system init messages in the iteration loop by checking message type
**Changes**:
- Lines 290-298: Removed callback options, kept valid options only
- Lines 300-312: Added iteration loop to handle system init messages
- The session ID is now captured when processing messages with `type === 'system' && subtype === 'init'`
### 3. Line 289 (now removed): Parameter 'msg' implicitly has 'any' type
**Error**: The callback parameter didn't have a type annotation.
**Fix**: This error was resolved by removing the invalid callback entirely (see fix #2).
### 4. Line 300 (now removed): Parameter 'msg' implicitly has 'any' type
**Error**: The callback parameter didn't have a type annotation.
**Fix**: This error was resolved by removing the invalid callback entirely (see fix #2).
### 5. Line 380 (now 404): Argument type error for Observation - missing `id` and `created_at_epoch`
**Error**: The `buildObservationPrompt()` function expects an `Observation` type with `id` and `created_at_epoch` properties, but the code was only passing `tool_name`, `tool_input`, and `tool_output`.
**Fix**:
- Added the missing `id: 0` (with comment explaining it's not needed for prompt generation)
- Added `created_at_epoch: Date.now()` to provide the current timestamp
**Changes**:
- Lines 404-410: Complete Observation object with all required properties
### 6. Line 527 (now 555): Property 'main' does not exist on type 'ImportMeta'
**Error**: TypeScript's default `ImportMeta` interface doesn't include Bun's custom `main` property.
**Fix**:
- Added a global type declaration to extend the `ImportMeta` interface with Bun's `main` property
- Used TypeScript's declaration merging to add the property type-safely
**Changes**:
- Lines 7-12: Added global declaration block extending `ImportMeta` with `main: boolean`
## Additional Changes
### Import Updates
- Line 17: Added import of `SDKUserMessage` and `SDKSystemMessage` types from the SDK package
### SDK Message Handling
- Lines 300-331: Refactored from callback-based approach to iteration-based approach
- Added proper message type checking and handling for both system and assistant messages
- Added content extraction logic for assistant messages (lines 316-320) to handle both array and string content types
## Verification
All TypeScript errors have been resolved:
- ✅ AsyncIterable type now matches SDK expectations
- ✅ No invalid callback options used
- ✅ All parameters have explicit types
- ✅ Observation objects have all required properties
- ✅ ImportMeta.main property is properly typed for Bun
## Logging Preservation
All logging statements have been preserved:
- ✅ All `console.error()` statements remain intact
- ✅ Debug logging for socket operations preserved
- ✅ Worker lifecycle logging preserved
- ✅ Message processing logging preserved
- ✅ SDK agent interaction logging preserved
The refactoring from callbacks to iteration actually improved logging by making the message handling flow more explicit and easier to follow.
docs/plans/phase0-task3-summary.md
-275
View File
@@ -1,275 +0,0 @@
# Phase 0 Task 3: Context Hook Logging Implementation
## Summary
Added comprehensive logging to the context hook (`src/hooks/context.ts`) to verify it correctly loads summaries from the database and outputs them as Claude's context. All logging uses `console.error` to avoid polluting stdout, which is reserved for the markdown context output that becomes part of Claude's context.
## Files Modified
- `/Users/alexnewman/Scripts/claude-mem/src/hooks/context.ts`
## Logging Points Added
All log messages use the `[claude-mem context]` prefix for easy searching and filtering.
### 1. Hook Invocation (Line 18-23)
```typescript
console.error('[claude-mem context] Hook fired with input:', JSON.stringify({
session_id: input?.session_id,
cwd: input?.cwd,
source: input?.source,
has_input: !!input
}));
```
**Purpose:** Logs that the hook was called and shows the input parameters, especially the `source` field which determines if context should be loaded.
### 2. Standalone Mode Detection (Line 27)
```typescript
console.error('[claude-mem context] No input provided - exiting (standalone mode)');
```
**Purpose:** Logs when the hook is run standalone without Claude Code input.
### 3. Source Check - Skip (Line 34)
```typescript
console.error('[claude-mem context] Source is not "startup" (got:', input.source, ') - skipping context load');
```
**Purpose:** Logs when the source is not "startup" (e.g., "resume"), indicating context loading is being skipped.
### 4. Source Check - Proceed (Line 39)
```typescript
console.error('[claude-mem context] Source check passed - proceeding with context load');
```
**Purpose:** Confirms we're proceeding with context loading because source check passed.
### 5. Project Extraction (Line 43)
```typescript
console.error('[claude-mem context] Extracted project name:', project, 'from cwd:', input.cwd);
```
**Purpose:** Shows the project name extracted from the cwd, which is used to query summaries.
### 6. Database Query Start (Line 46)
```typescript
console.error('[claude-mem context] Querying database for recent summaries...');
```
**Purpose:** Indicates we're about to query the database.
### 7. Database Query Results (Line 51)
```typescript
console.error('[claude-mem context] Database query complete - found', summaries.length, 'summaries');
```
**Purpose:** Reports how many summaries were found in the database.
### 8. Summary Previews (Lines 54-60)
```typescript
if (summaries.length > 0) {
console.error('[claude-mem context] Summary previews:');
summaries.forEach((summary, idx) => {
const preview = summary.request?.substring(0, 100) || summary.completed?.substring(0, 100) || '(no content)';
console.error(` [${idx + 1}]`, preview + (preview.length >= 100 ? '...' : ''));
});
}
```
**Purpose:** Shows a preview (first 100 chars) of each summary found, helping verify the correct data was retrieved.
### 9. No Summaries Found (Line 64)
```typescript
console.error('[claude-mem context] No summaries found - outputting empty context message');
```
**Purpose:** Logs when no summaries exist for the project.
### 10. Markdown Building Start (Line 70)
```typescript
console.error('[claude-mem context] Building markdown context from summaries...');
```
**Purpose:** Indicates we're starting to build the markdown output.
### 11. Markdown Output Details (Lines 117-120)
```typescript
console.error('[claude-mem context] Markdown built successfully');
console.error('[claude-mem context] Output length:', markdownOutput.length, 'characters,', output.length, 'lines');
console.error('[claude-mem context] Output preview (first 200 chars):', markdownOutput.substring(0, 200) + '...');
console.error('[claude-mem context] Outputting context to stdout for Claude Code injection');
```
**Purpose:** Reports the markdown was built successfully, shows its length, and provides a preview before sending to stdout.
### 12. Successful Completion (Line 125)
```typescript
console.error('[claude-mem context] Context hook completed successfully');
```
**Purpose:** Confirms the hook completed without errors.
### 13. Error Handling (Lines 130-133)
```typescript
console.error('[claude-mem context] ERROR occurred during context hook execution');
console.error('[claude-mem context] Error message:', error.message);
console.error('[claude-mem context] Error stack:', error.stack);
console.error('[claude-mem context] Exiting gracefully to avoid blocking Claude Code');
```
**Purpose:** Provides detailed error information if anything goes wrong, including stack trace for debugging.
## Critical Implementation Detail: stdout vs stderr
**IMPORTANT:** All logging uses `console.error` (stderr) because:
- The context hook outputs markdown to `console.log` (stdout)
- Claude Code reads stdout to inject context into Claude's conversation
- Any logging to stdout would pollute the context and break the feature
- stderr is safe for logging and will appear in Claude Code's logs/terminal
## How to Test
### Testing with an Existing Project with Summaries
1. **Ensure you have previous summaries saved:**
```bash
# Check if summaries exist for your project
sqlite3 ~/.config/claude-code/hooks/claude-mem.db "SELECT * FROM summaries WHERE project = 'your-project-name' LIMIT 5;"
```
2. **Start a new Claude Code session:**
```bash
cd /path/to/your-project
claude-code
```
3. **Check the logs:**
- Look for `[claude-mem context]` messages in stderr
- Claude Code should show these logs during startup
- The context should appear in Claude's initial knowledge
### Testing with a New Project (No Summaries)
1. **Navigate to a project without previous summaries:**
```bash
cd /path/to/new-project
claude-code
```
2. **Expected behavior:**
- Hook fires and logs indicate no summaries found
- Output should be: "No previous sessions found for this project yet."
### Testing Standalone Mode
```bash
# Run the hook directly (not via Claude Code)
tsx src/hooks/context.ts
# Expected output:
# [claude-mem context] Hook fired with input: {...}
# [claude-mem context] No input provided - exiting (standalone mode)
# No input provided - this script is designed to run as a Claude Code SessionStart hook
```
### Testing Source Check (Resume vs Startup)
The hook should only load context on `source: "startup"`, not on session resume. This is harder to test directly but the logs will show:
- On startup: "Source check passed - proceeding with context load"
- On resume: "Source is not 'startup' (got: resume) - skipping context load"
## Expected Log Sequence for a Session with Previous Summaries
When you start Claude Code in a project with existing summaries, you should see this sequence in stderr:
```
[claude-mem context] Hook fired with input: {"session_id":"...","cwd":"/path/to/project","source":"startup","has_input":true}
[claude-mem context] Source check passed - proceeding with context load
[claude-mem context] Extracted project name: project from cwd: /path/to/project
[claude-mem context] Querying database for recent summaries...
[claude-mem context] Database query complete - found 3 summaries
[claude-mem context] Summary previews:
[1] Added logging to the save hook to track when summaries are being persisted to the database...
[2] Implemented the worker hook to generate summaries from session transcripts using Claude API...
[3] Created database schema and initial setup for storing session summaries...
[claude-mem context] Building markdown context from summaries...
[claude-mem context] Markdown built successfully
[claude-mem context] Output length: 1247 characters, 45 lines
[claude-mem context] Output preview (first 200 chars): # Recent Session Context
Here's what happened in recent project sessions:
---
**Request:** Added logging to the save hook to track when summaries are being persisted to the database
**Completed:** ...
[claude-mem context] Outputting context to stdout for Claude Code injection
[claude-mem context] Context hook completed successfully
```
## What to Look For in Logs
### Success Indicators
1. Hook fires with `has_input: true` and `source: "startup"`
2. Source check passes
3. Project name is correctly extracted
4. Database query finds summaries (count > 0)
5. Summary previews show meaningful content
6. Markdown is built with reasonable length (> 100 characters)
7. Hook completes successfully
### Warning Signs
1. Hook fires with `has_input: false` - means Claude Code didn't provide input
2. Source is not "startup" - context won't load (expected on resume)
3. Database query finds 0 summaries - either first session or save hook not working
4. Summary previews show "(no content)" - data might be corrupt
5. Markdown length is very small - formatting might be broken
6. Error messages appear - check stack trace for issues
### Common Issues to Debug
**No summaries found:**
- Check if save hook is configured and working
- Verify worker hook generated summaries
- Ensure project name matches (case-sensitive)
**Hook doesn't fire:**
- Verify hooks are configured in Claude Code settings
- Check that the hook path is correct
- Ensure the built JavaScript exists (`dist/hooks/context.js`)
**Context not appearing in Claude:**
- Check if markdown is being output to stdout (should see in logs)
- Verify stdout isn't being polluted by other logs
- Check Claude Code configuration for SessionStart hooks
## Issues or Concerns Discovered
### None - Implementation is Clean
The implementation is straightforward and follows best practices:
1. **Separation of concerns:** stdout for context, stderr for logging
2. **Comprehensive coverage:** Every critical step is logged
3. **Safe error handling:** Errors are logged but don't block Claude Code
4. **No performance impact:** Logging is lightweight
5. **Easy debugging:** All logs are prefixed and searchable
### Future Enhancements (Optional)
1. **Log levels:** Could add debug/info/error levels for filtering
2. **Timing information:** Could log how long database queries take
3. **Conditional logging:** Could enable/disable via environment variable
4. **Structured logging:** Could output logs as JSON for parsing
## Testing Checklist
- [ ] Start Claude Code in a project with existing summaries
- [ ] Verify logs appear in stderr with `[claude-mem context]` prefix
- [ ] Confirm context appears in Claude's initial knowledge
- [ ] Check summary previews match actual summary content
- [ ] Verify markdown length is reasonable
- [ ] Test with a new project (no summaries)
- [ ] Confirm "No previous sessions found" message appears
- [ ] Run hook standalone and verify it exits gracefully
- [ ] Check that all log points are hit in sequence
- [ ] Verify no logs appear in stdout (only markdown context)
## Conclusion
The context hook now has comprehensive logging at every critical step. This will make it easy to:
- Verify summaries are being loaded from the database
- Debug issues with context not appearing
- Confirm the markdown output is correct
- Track the complete flow from hook invocation to Claude context injection
All logging uses stderr to avoid polluting the stdout channel that carries the actual context markdown to Claude Code.
docs/plans/phase0-task4-summary.md
-279
View File
@@ -1,279 +0,0 @@
# Phase 0 Task 4 Summary: Pre-Test Diagnostics
**Date:** 2025-10-16
**Task:** Verify logging changes and prepare end-to-end test plan
---
## Diagnostics Performed
### 1. Compiled Hook File Verification
Checked three compiled JavaScript files to verify logging survived the build process:
**Files Checked:**
- `/Users/alexnewman/Scripts/claude-mem/scripts/hooks/summary-hook.js` (4.6K)
- `/Users/alexnewman/Scripts/claude-mem/scripts/hooks/context-hook.js` (5.8K)
- `/Users/alexnewman/Scripts/claude-mem/scripts/hooks/worker.js` (238K)
**Results:**
- summary-hook.js: Contains 3 instances of `[claude-mem summary]` logging
- context-hook.js: Contains 3 instances of `[claude-mem context]` logging
- worker.js: Contains multiple instances of `[claude-mem worker]` logging
**Status:** PASS - All logging statements are present in compiled files
### 2. Database State Analysis
Queried the claude-mem database to understand current state:
**Database Location:** `~/.claude-mem/claude-mem.db`
**Findings:**
- Total SDK sessions recorded: 37
- Active sessions: 0
- Completed sessions: 22
- Failed sessions: 0 (inferred)
- Session summaries: 0
**Recent Sessions:**
```
ID 37: completed at 2025-10-16T21:39:18.888Z, project: claude-mem
ID 36: completed at 2025-10-16T21:24:30.850Z, project: claude-mem
ID 35: completed at 2025-10-16T21:11:12.929Z, project: claude-mem-test
ID 34: completed at 2025-10-16T20:59:43.438Z, project: claude-mem-test
ID 33: completed at 2025-10-16T20:55:15.426Z, project: claude-mem-test
```
**Database Tables Present:**
- diagnostics
- memories
- observations
- overviews
- schema_versions
- sdk_sessions (properly indexed)
- session_locks
- session_summaries
- sessions
- sqlite_sequence
- transcript_events
**Status:** Database structure is correct, but summary generation appears to have issues
### 3. Hooks Configuration Verification
Checked the Claude Code hooks configuration:
**Hooks File Location:** `/Users/alexnewman/Scripts/claude-mem/hooks/hooks.json`
**Configured Hooks:**
- SessionStart: Runs `context-hook.js` to inject previous session context
- UserPromptSubmit: Runs `new-hook.js` to create SDK session and spawn worker
- PostToolUse: Runs `save-hook.js` to record tool observations
- Stop: Runs `summary-hook.js` to finalize session and generate summary
**Status:** All hooks properly configured with appropriate timeouts
### 4. Worker Process Check
Checked for running worker processes and socket files:
**Commands Used:**
```bash
ps aux | grep claude-mem-worker | grep -v grep
ls -la /tmp/claude-mem-worker-*.sock
```
**Results:**
- No running worker processes detected
- No socket files found in /tmp/
**Status:** Clean slate - no zombie workers or stale sockets
### 5. Test Plan Creation
Created comprehensive test plan document at:
`/Users/alexnewman/Scripts/claude-mem/docs/plans/phase0-test-plan.md`
**Contents:**
- Pre-test checklist with current system state
- Step-by-step test execution instructions
- Expected log sequences for each component
- Log collection and filtering commands
- Success criteria checklist
- Troubleshooting guide
---
## Current State of the System
### Overall Health: READY FOR TESTING
The system is in a clean state with no active sessions or running workers. Logging is confirmed to be present in all compiled hook files.
### Component Status
| Component | Status | Notes |
|-----------|--------|-------|
| summary-hook.js | READY | Logging present, executable, configured in hooks.json |
| context-hook.js | READY | Logging present, executable, configured in hooks.json |
| new-hook.js | READY | Executable, configured in hooks.json |
| save-hook.js | READY | Executable, configured in hooks.json |
| worker.js | READY | Logging present, executable |
| Database | READY | Clean, no active sessions |
| Worker processes | CLEAN | No running workers |
| Socket files | CLEAN | No stale sockets |
| Hooks configuration | READY | All lifecycle events properly configured |
### File Permissions
All hook files have execute permissions:
```
-rwxr-xr-x context-hook.js
-rwxr-xr-x new-hook.js
-rwxr-xr-x save-hook.js
-rwxr-xr-x summary-hook.js
-rwxr-xr-x worker.js
```
---
## Issues Found
### Critical Issue: Zero Summaries Despite Completed Sessions
**Severity:** HIGH
**Description:** The database shows 22 completed SDK sessions but 0 session_summaries. This suggests the summary generation pipeline may not be working correctly.
**Possible Causes:**
1. Worker may not be receiving FINALIZE messages
2. SDK agent may not be responding with expected XML format
3. Summary parsing may be failing silently
4. Database write may be failing
**Impact:** This is the core functionality we're testing - summaries must be generated for context to work
**Next Steps:** The end-to-end test will help diagnose where in the pipeline the failure occurs
### Minor Issue: Multiple Database Files
**Severity:** LOW
**Description:** Multiple database files found in ~/.claude-mem/:
- memories.db
- claude-mem.db
- index.db
- memory.db
- hooks.db
**Impact:** Potential confusion about which database is active. Code appears to use `~/.claude-mem/claude-mem.db`
**Recommendation:** Clean up old/unused database files after confirming current one is correct
---
## Logging Implementation Verification
### Summary Hook Logging
Located in compiled `summary-hook.js` at multiple points:
1. Hook entry point: "Hook fired"
2. Session search: "Searching for active SDK session"
3. Session found: "Active SDK session found"
4. Socket operations: "Attempting to send FINALIZE message", "Socket connection established"
5. Completion: "Socket connection closed successfully"
### Context Hook Logging
Located in compiled `context-hook.js` at multiple points:
1. Hook entry: "Hook fired with input:"
2. Source validation: "Source check passed"
3. Project extraction: "Extracted project name"
4. Database query: "Querying database for recent summaries..."
5. Results: "Database query complete - found X summaries"
6. Markdown generation: "Building markdown context from summaries..."
7. Completion: "Context hook completed successfully"
### Worker Logging
Located in compiled `worker.js` throughout the lifecycle:
1. Instance creation: "Worker instance created"
2. Session loading: "Session loaded successfully"
3. Socket server: "Socket server started successfully"
4. SDK agent: "Starting SDK agent", "SDK session initialized"
5. Message handling: "Message received from socket"
6. Summary parsing: "Summary parsed successfully", "Storing summary in database"
7. Cleanup: "Cleaning up worker resources"
---
## Recommendations for Next Steps
### Immediate: Run End-to-End Test
1. Follow the test plan in `phase0-test-plan.md`
2. Capture all logs (redirect stderr to file)
3. Pay special attention to summary generation
4. Verify each success criterion
### Priority: Investigate Summary Generation Failure
The zero summaries issue needs immediate attention:
1. Check if workers are being spawned by new-hook.js
2. Verify SDK agent responses include expected XML
3. Add more detailed logging in summary parsing
4. Check database write permissions and constraints
### Monitoring During Test
Watch these areas closely:
1. Worker process spawning (should happen in new-hook)
2. Socket creation in /tmp/
3. FINALIZE message delivery
4. Summary parsing and storage
5. Context injection in second session
### After Test
1. Document all findings from test execution
2. Collect and analyze all logs
3. Update code to fix any issues found
4. Consider adding automated tests
5. Update documentation based on learnings
---
## Test Environment Details
**Operating System:** macOS (Darwin 25.0.0)
**Working Directory:** /Users/alexnewman/Scripts/claude-mem
**Git Branch:** feature/source-repo
**Database Path:** ~/.claude-mem/claude-mem.db
**Socket Path Pattern:** /tmp/claude-mem-worker-{sessionId}.sock
**Hook Directory:** /Users/alexnewman/Scripts/claude-mem/scripts/hooks/
**Claude Code Configuration:**
- Config directory: ~/.claude/
- Project hooks file: /Users/alexnewman/Scripts/claude-mem/hooks/hooks.json
- Hooks properly configured for all lifecycle events:
- SessionStart: context-hook.js (180s timeout)
- UserPromptSubmit: new-hook.js (60s timeout)
- PostToolUse: save-hook.js (180s timeout)
- Stop: summary-hook.js (60s timeout)
---
## Deliverables
1. **Test Plan Document:** `/Users/alexnewman/Scripts/claude-mem/docs/plans/phase0-test-plan.md`
- Comprehensive testing instructions
- Success criteria
- Log collection commands
- Troubleshooting guide
2. **This Summary Document:** `/Users/alexnewman/Scripts/claude-mem/docs/plans/phase0-task4-summary.md`
- Diagnostic results
- System state analysis
- Issues identified
- Recommendations
3. **Pre-Test Validation:** COMPLETE
- Logging verified in all compiled files
- Database state documented
- Worker state confirmed clean
- System ready for testing
---
## Conclusion
The system is ready for end-to-end testing. All logging has successfully survived the build process and is present in the compiled hook files. The database is in a clean state with no active sessions or zombie workers.
However, the zero summaries despite 22 completed sessions is a critical issue that the end-to-end test should help diagnose. The test plan provides detailed instructions for execution, log collection, and success verification.
**Status:** READY TO PROCEED with end-to-end testing
**Next Action:** Execute the test plan in `phase0-test-plan.md` and collect all logs for analysis
docs/plans/phase0-test-plan.md
-310
View File
@@ -1,310 +0,0 @@
# Phase 0 End-to-End Test Plan
## Overview
This test plan validates the complete claude-mem pipeline from session start through context injection in a new session. The test verifies that logging, worker processes, database updates, and context retrieval all function correctly.
---
## Section 1: Pre-Test Checklist
### Current Database State (as of 2025-10-16)
- **Database Location:** `~/.claude-mem/claude-mem.db`
- **Total SDK Sessions:** 37 sessions recorded
- **Active Sessions:** 0 (all sessions properly closed)
- **Completed Sessions:** 22
- **Session Summaries:** 0 (ISSUE: No summaries despite completed sessions)
- **Recent Sessions:**
```
ID 37: completed at 2025-10-16T21:39:18.888Z, project: claude-mem
ID 36: completed at 2025-10-16T21:24:30.850Z, project: claude-mem
ID 35: completed at 2025-10-16T21:11:12.929Z, project: claude-mem-test
ID 34: completed at 2025-10-16T20:59:43.438Z, project: claude-mem-test
ID 33: completed at 2025-10-16T20:55:15.426Z, project: claude-mem-test
```
### Current Worker State
- **Running Workers:** None detected
- **Socket Files:** No active sockets in /tmp/
- **Command Used:** `ps aux | grep claude-mem-worker | grep -v grep`
### Logging Verification in Compiled Files
- **summary-hook.js:** Contains 3 instances of `[claude-mem summary]` logging
- **context-hook.js:** Contains 3 instances of `[claude-mem context]` logging
- **worker.js:** Contains multiple instances of `[claude-mem worker]` logging
- **Status:** CONFIRMED - All logging survived the build process
### Pre-Test Issues Identified
1. Zero session_summaries despite 22 completed SDK sessions - suggests summary generation may not be working
2. No active workers or sockets - clean state for testing
---
## Section 2: Test Execution Steps
### Step 1: Clean Slate (Optional - if you want to start fresh)
```bash
# Backup current database
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/backups/claude-mem-backup-$(date +%Y%m%d-%H%M%S).db
# Optional: Clear old sessions if desired
# sqlite3 ~/.claude-mem/claude-mem.db "DELETE FROM sdk_sessions WHERE status = 'completed'"
# sqlite3 ~/.claude-mem/claude-mem.db "DELETE FROM session_summaries"
```
### Step 2: Start Claude Code Session 1
```bash
# Navigate to the test project
cd /Users/alexnewman/Scripts/claude-mem
# Start Claude Code
# Logs will show context-hook.js firing
# Expected: "[claude-mem context] Hook fired with input:"
claude
```
### Step 3: Do Some Work in Session 1
Within the Claude Code session, ask Claude to perform meaningful work:
```
Please help me:
1. Read the README.md file
2. Analyze the project structure
3. List the main TypeScript files in src/
4. Create a simple test file at test/example.test.ts with a placeholder test
```
Wait for Claude to complete all tasks.
### Step 4: Exit Session 1
```bash
# Type exit or Ctrl+D to end the session
# Expected: summary-hook.js will fire
# Expected: "[claude-mem summary] Hook fired" message
# Expected: Worker will process and generate summary
exit
```
### Step 5: Check Database for Summary
```bash
# Wait 5-10 seconds for worker to complete processing
sleep 10
# Check if a new SDK session was created
sqlite3 ~/.claude-mem/claude-mem.db "SELECT id, status, started_at, project FROM sdk_sessions ORDER BY started_at_epoch DESC LIMIT 3"
# Check if summary was generated
sqlite3 ~/.claude-mem/claude-mem.db "SELECT id, request, completed, created_at FROM session_summaries ORDER BY created_at_epoch DESC LIMIT 1"
# Check observations
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations WHERE sdk_session_id = (SELECT sdk_session_id FROM sdk_sessions ORDER BY started_at_epoch DESC LIMIT 1)"
```
### Step 6: Start Claude Code Session 2
```bash
# Start a new session in the same project
cd /Users/alexnewman/Scripts/claude-mem
claude
```
### Step 7: Ask Claude About Previous Session
Within Session 2, ask:
```
What did we work on in the previous session? What files were modified?
```
Claude should reference the previous session context that was injected.
### Step 8: Collect All Logs
```bash
# Exit session 2
exit
# Collect logs (location depends on your Claude Code setup)
# Check stderr output from both sessions
# Filter for claude-mem messages
```
---
## Section 3: What to Look For
### Expected Log Sequence from Summary Hook
```
[claude-mem summary] Hook fired
[claude-mem summary] Searching for active SDK session
[claude-mem summary] Active SDK session found
[claude-mem summary] Attempting to send FINALIZE message to worker socket
[claude-mem summary] Socket connection established, sending message
[claude-mem summary] Socket connection closed successfully
```
### Expected Log Sequence from Worker
```
[claude-mem worker] Worker instance created
[claude-mem worker] Worker run() started
[claude-mem worker] Session loaded successfully
[claude-mem worker] Socket server started successfully
[claude-mem worker] Starting SDK agent
[claude-mem worker] SDK session initialized
[claude-mem worker] SDK agent response received
[claude-mem worker] Parsing agent message for observations and summary
[claude-mem worker] Summary parsed successfully
[claude-mem worker] Storing summary in database
[claude-mem worker] Summary stored successfully in database
[claude-mem worker] SDK agent completed, marking session as completed
[claude-mem worker] Cleaning up worker resources
```
### Expected Log Sequence from Context Hook
```
[claude-mem context] Hook fired with input:
[claude-mem context] Source check passed - proceeding with context load
[claude-mem context] Extracted project name: claude-mem from cwd: /Users/alexnewman/Scripts/claude-mem
[claude-mem context] Querying database for recent summaries...
[claude-mem context] Database query complete - found X summaries
[claude-mem context] Building markdown context from summaries...
[claude-mem context] Markdown built successfully
[claude-mem context] Outputting context to stdout for Claude Code injection
[claude-mem context] Context hook completed successfully
```
### How to Verify Summary in Database
After Session 1 exits, the summary should contain:
- **request:** Description of what was asked
- **investigated:** Files/areas examined
- **learned:** Key findings
- **completed:** What was accomplished
- **next_steps:** Recommendations
- **files_read:** JSON array of files read
- **files_edited:** JSON array of files modified (should include test/example.test.ts)
### How to Verify Context Was Loaded
In Session 2:
1. Claude should reference the previous session without being told
2. The context-hook.js logs should show summaries were found and loaded
3. Claude's response should mention specific files or tasks from Session 1
---
## Section 4: Log Collection Commands
### Filter Logs for Summary Hook
```bash
# From Claude Code stderr output
grep "\[claude-mem summary\]" ~/.claude-code/logs/*.log 2>/dev/null || echo "Check your Claude Code log location"
# Alternative: redirect stderr during session
claude 2>&1 | tee /tmp/claude-session.log
# Then: grep "\[claude-mem summary\]" /tmp/claude-session.log
```
### Filter Logs for Context Hook
```bash
grep "\[claude-mem context\]" /tmp/claude-session.log
```
### Filter Logs for Worker
```bash
grep "\[claude-mem worker\]" /tmp/claude-session.log
```
### Search for Errors
```bash
# Search for any errors in the logs
grep -i "error\|fail\|exception" /tmp/claude-session.log | grep claude-mem
# Check for database errors
grep "sqlite\|database" /tmp/claude-session.log | grep -i error
```
### Verify Each Step of the Pipeline
```bash
# 1. Verify session was created
sqlite3 ~/.claude-mem/claude-mem.db "SELECT * FROM sdk_sessions WHERE id = (SELECT MAX(id) FROM sdk_sessions)"
# 2. Verify worker socket was created (during session)
ls -la /tmp/claude-mem-worker-*.sock
# 3. Verify observations were recorded
sqlite3 ~/.claude-mem/claude-mem.db "SELECT type, text FROM observations WHERE sdk_session_id = (SELECT sdk_session_id FROM sdk_sessions ORDER BY started_at_epoch DESC LIMIT 1)"
# 4. Verify summary was created
sqlite3 ~/.claude-mem/claude-mem.db "SELECT request, completed, files_edited FROM session_summaries ORDER BY created_at_epoch DESC LIMIT 1"
```
### Monitor Worker Process
```bash
# During session, check if worker is running
watch -n 1 "ps aux | grep claude-mem-worker | grep -v grep"
# Check worker socket
watch -n 1 "ls -la /tmp/claude-mem-worker-*.sock 2>/dev/null"
```
---
## Section 5: Success Criteria
### Must Pass (Critical)
- [ ] Session 1 creates an entry in sdk_sessions with status='active'
- [ ] Context hook fires at Session 1 start and logs show it ran
- [ ] Summary hook fires at Session 1 exit and logs show it ran
- [ ] Worker process starts and creates a socket file
- [ ] Worker receives FINALIZE message from summary hook
- [ ] Summary is successfully parsed and stored in session_summaries table
- [ ] Session status changes from 'active' to 'completed'
- [ ] Socket file is cleaned up after worker exits
- [ ] Session 2 starts and context hook fires
- [ ] Context hook finds summaries and injects them as markdown
- [ ] Claude references previous session in Session 2
### Should Pass (Important)
- [ ] Observations are recorded in the observations table
- [ ] files_read and files_edited are populated in summary
- [ ] No error messages in logs
- [ ] Worker process exits cleanly
- [ ] No zombie workers or stale sockets remain
### Nice to Have
- [ ] All log messages are clear and informative
- [ ] Timing is reasonable (summary generation < 30 seconds)
- [ ] Multiple sessions can be loaded in context
- [ ] Context markdown is well-formatted
### Known Issues to Monitor
- [ ] Zero summaries in current database despite 22 completed sessions - needs investigation
- [ ] Verify worker is actually spawned (new-hook.js responsible for this)
- [ ] Confirm SDK session ID is properly set
---
## Troubleshooting Guide
### If Summary Hook Doesn't Fire
1. Check that hooks are properly configured in ~/.claude/hooks.json
2. Verify summary-hook.js has execute permissions
3. Check Claude Code version supports hooks
### If Worker Doesn't Start
1. Check new-hook.js logs - it should spawn the worker
2. Verify worker.js has execute permissions
3. Check for port/socket conflicts
### If Summary Is Not Generated
1. Check worker logs for parsing errors
2. Verify SDK agent is responding with expected XML format
3. Check database write permissions
### If Context Doesn't Load
1. Verify summaries exist in database
2. Check context-hook.js logs for query results
3. Verify project name extraction is correct
---
## Next Steps After Test
1. If test passes: Proceed to Phase 1 (advanced features)
2. If test fails: Collect all logs and diagnostic info
3. Document any issues found
4. Update code as needed
5. Re-run test until success criteria met
docs/plans/session-logic-fixes.md
-1123
View File
File diff suppressed because it is too large Load Diff