Files
claude-mem/README.md
T
Alex Newman 36b0929fae Server-beta: Postgres storage + independent runtime + BullMQ queue (Phases 1–3) (#2351)
* Add server beta runtime foundation

* Address server beta review findings

* Resolve server beta review comments

* Tighten server beta review follow-ups

* Harden server beta auth and search

* Avoid unnecessary FTS rebuilds

* Block scoped keys from creating projects

* Release BullMQ claims best effort on close

* Address server beta review blockers

* Reset BullMQ claims best effort

* Add Postgres observation storage foundation

* feat(server-beta): add independent runtime service

Introduce src/server/runtime/ as a self-contained server-beta runtime
that owns its lifecycle, Postgres bootstrap, and HTTP boundary without
depending on WorkerService.

ServerBetaService wraps the existing Server class, exposes
/healthz and /v1/info with runtime="server-beta", and persists state
to dedicated paths (.server-beta.pid|.port|.runtime.json). The four
boundary managers (queue, generation worker, provider registry, event
broadcaster) are intentionally disabled in this phase and report their
status through /v1/info; later phases activate them.

Adds plans/2026-05-07-finish-bullmq-branch-ship-plan.md to track the
remaining work for this branch.

Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): route CLI lifecycle and bundle separate runtime

scripts/build-hooks.js now produces plugin/scripts/server-beta-service.cjs
as a separate Node CJS bundle, alongside the existing worker-service
bundle. The server-beta runtime is now installable independently.

src/npx-cli/commands/server.ts routes start|stop|restart|status to the
server-beta lifecycle instead of the legacy worker. The worker keeps its
own start|stop|restart|status under the worker namespace; the two
runtimes can be operated independently.

src/services/worker-service.ts adds a server-* command parser branch
that delegates to the sibling server-beta-service.cjs bundle so
direct worker-service invocations still route to the right runtime.

tests/npx-cli-server-namespace.test.ts updated to expect server-beta
lifecycle routing.

Includes rebuilt plugin/scripts/*.cjs bundles produced by
build-and-sync.

Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): add BullMQ job queue primitives

Introduce src/server/jobs/ as the queue-side primitives that Phase 3 of
the server-beta runtime needs to operate.

types.ts defines a discriminated union over the four job kinds (event,
event-batch, summary, reindex) and maps each to a per-kind BullMQ queue
name and deterministic-ID prefix.

job-id.ts builds deterministic, colon-free BullMQ jobIds from
(kind, team, project, source). The colon ban exists because BullMQ uses
':' as a Redis key separator internally; embedding ':' in jobIds
breaks scan and state lookups.

ServerJobQueue.ts is a thin wrapper over BullMQ Queue + Worker that
enforces autorun:false, default concurrency 1, and an attached error
listener — all per BullMQ docs requirements. Test seams accept queue
and worker factories so unit tests do not need Redis.

outbox.ts publishes through the Postgres ObservationGenerationJob
repository as canonical history. enqueueOutbox writes the row first,
then publishes to BullMQ; if BullMQ throws, the row is transitioned to
failed and a failed event is appended. reconcileOnStartup re-enqueues
queued + processing rows after a restart, replacing terminal BullMQ
jobs that may still be holding the deterministic ID slot. markCompleted
and markFailed wrap transitionStatus and append the matching event row.

Includes 20 unit tests covering deterministic ID stability, colon-free
output, queue lifecycle, error-listener attachment, double-start
refusal, idempotent enqueue, BullMQ failure rollback, startup
reconciliation, max-attempts skipping, and completion / failure /
retry transitions.

Phase 3 commit 1 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): activate queue boundary in runtime service

Wire ActiveServerBetaQueueManager into the server-beta runtime graph.
The active manager owns one ServerJobQueue per generation kind (event,
event-batch, summary, reindex) and surfaces lane metadata through
boundary health.

Selection is opt-in and fail-fast: if CLAUDE_MEM_QUEUE_ENGINE is set to
bullmq the active manager is constructed (and any Redis/config error
throws — no silent fallback to SQLite, per Phase 3 anti-pattern guard).
For any other engine the disabled boundary remains so worker-era and
test setups stay compatible.

Widens ServerBetaBoundaryHealth.status to a discriminated union
('disabled' | 'active' | 'errored') with optional details. The disabled
adapter still emits status='disabled', which keeps the existing
server-beta-service test green.

ServerBetaService receives the manager through a new optional
queueManager field on CreateServerBetaServiceOptions so test graphs
and Phase 4 wiring can inject custom managers.

Adds tests/server/runtime/active-queue-manager.test.ts covering bullmq
guard, active health shape, per-kind queue access, close behavior, and
post-close errored health.

Phase 3 commit 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(server-beta): cap /v1/events/batch at 500 events

Prevents unbounded array DoS surface flagged in PR review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 01:20:07 -07:00

420 lines
17 KiB
Markdown

<h1 align="center">
<br>
<a href="https://github.com/thedotmack/claude-mem">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/claude-mem-logo-for-dark-mode.webp">
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/claude-mem-logo-for-light-mode.webp">
<img src="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/claude-mem-logo-for-light-mode.webp" alt="Claude-Mem" width="400">
</picture>
</a>
<br>
</h1>
<p align="center">
<a href="docs/i18n/README.zh.md">🇨🇳 中文</a> •
<a href="docs/i18n/README.zh-tw.md">🇹🇼 繁體中文</a> •
<a href="docs/i18n/README.ja.md">🇯🇵 日本語</a> •
<a href="docs/i18n/README.pt.md">🇵🇹 Português</a> •
<a href="docs/i18n/README.pt-br.md">🇧🇷 Português</a> •
<a href="docs/i18n/README.ko.md">🇰🇷 한국어</a> •
<a href="docs/i18n/README.es.md">🇪🇸 Español</a> •
<a href="docs/i18n/README.de.md">🇩🇪 Deutsch</a> •
<a href="docs/i18n/README.fr.md">🇫🇷 Français</a> •
<a href="docs/i18n/README.he.md">🇮🇱 עברית</a> •
<a href="docs/i18n/README.ar.md">🇸🇦 العربية</a> •
<a href="docs/i18n/README.ru.md">🇷🇺 Русский</a> •
<a href="docs/i18n/README.pl.md">🇵🇱 Polski</a> •
<a href="docs/i18n/README.cs.md">🇨🇿 Čeština</a> •
<a href="docs/i18n/README.nl.md">🇳🇱 Nederlands</a> •
<a href="docs/i18n/README.tr.md">🇹🇷 Türkçe</a> •
<a href="docs/i18n/README.uk.md">🇺🇦 Українська</a> •
<a href="docs/i18n/README.vi.md">🇻🇳 Tiếng Việt</a> •
<a href="docs/i18n/README.tl.md">🇵🇭 Tagalog</a> •
<a href="docs/i18n/README.id.md">🇮🇩 Indonesia</a> •
<a href="docs/i18n/README.th.md">🇹🇭 ไทย</a> •
<a href="docs/i18n/README.hi.md">🇮🇳 हिन्दी</a> •
<a href="docs/i18n/README.bn.md">🇧🇩 বাংলা</a> •
<a href="docs/i18n/README.ur.md">🇵🇰 اردو</a> •
<a href="docs/i18n/README.ro.md">🇷🇴 Română</a> •
<a href="docs/i18n/README.sv.md">🇸🇪 Svenska</a> •
<a href="docs/i18n/README.it.md">🇮🇹 Italiano</a> •
<a href="docs/i18n/README.el.md">🇬🇷 Ελληνικά</a> •
<a href="docs/i18n/README.hu.md">🇭🇺 Magyar</a> •
<a href="docs/i18n/README.fi.md">🇫🇮 Suomi</a> •
<a href="docs/i18n/README.da.md">🇩🇰 Dansk</a> •
<a href="docs/i18n/README.no.md">🇳🇴 Norsk</a>
</p>
<h4 align="center">Persistent memory compression system built for <a href="https://claude.com/claude-code" target="_blank">Claude Code</a>.</h4>
<p align="center">
<a href="LICENSE">
<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License">
</a>
<a href="package.json">
<img src="https://img.shields.io/badge/version-6.5.0-green.svg" alt="Version">
</a>
<a href="package.json">
<img src="https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg" alt="Node">
</a>
<a href="https://github.com/thedotmack/awesome-claude-code">
<img src="https://awesome.re/mentioned-badge.svg" alt="Mentioned in Awesome Claude Code">
</a>
</p>
<p align="center">
<a href="https://trendshift.io/repositories/15496" target="_blank">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/trendshift-badge-dark.svg">
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/trendshift-badge.svg">
<img src="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/trendshift-badge.svg" alt="thedotmack/claude-mem | Trendshift" width="250" height="55"/>
</picture>
</a>
</p>
<br>
<table align="center">
<tr>
<td align="center">
<a href="https://github.com/thedotmack/claude-mem">
<picture>
<img
src="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/cm-preview.gif"
alt="Claude-Mem Preview"
width="500"
>
</picture>
</a>
</td>
<td align="center">
<a href="https://www.star-history.com/#thedotmack/claude-mem&Date">
<picture>
<source
media="(prefers-color-scheme: dark)"
srcset="https://api.star-history.com/image?repos=thedotmack/claude-mem&type=date&theme=dark&legend=top-left"
/>
<source
media="(prefers-color-scheme: light)"
srcset="https://api.star-history.com/image?repos=thedotmack/claude-mem&type=date&legend=top-left"
/>
<img
alt="Star History Chart"
src="https://api.star-history.com/image?repos=thedotmack/claude-mem&type=date&legend=top-left"
width="500"
/>
</picture>
</a>
</td>
</tr>
</table>
<p align="center">
<a href="#quick-start">Quick Start</a> •
<a href="#how-it-works">How It Works</a> •
<a href="#mcp-search-tools">Search Tools</a> •
<a href="#documentation">Documentation</a> •
<a href="#configuration">Configuration</a> •
<a href="#troubleshooting">Troubleshooting</a> •
<a href="#license">License</a>
</p>
<p align="center">
Claude-Mem seamlessly preserves context across sessions by automatically capturing tool usage observations, generating semantic summaries, and making them available to future sessions. This enables Claude to maintain continuity of knowledge about projects even after sessions end or reconnect.
</p>
---
## Quick Start
Install with a single command:
```bash
npx claude-mem install
```
Or install for Gemini CLI (auto-detects `~/.gemini`):
```bash
npx claude-mem install --ide gemini-cli
```
Or install for OpenCode:
```bash
npx claude-mem install --ide opencode
```
Or install from the plugin marketplace inside Claude Code:
```bash
/plugin marketplace add thedotmack/claude-mem
/plugin install claude-mem
```
Restart Claude Code or Gemini CLI. Context from previous sessions will automatically appear in new sessions.
> **Note:** Claude-Mem is also published on npm, but `npm install -g claude-mem` installs the **SDK/library only** — it does not register the plugin hooks or set up the worker service. Always install via `npx claude-mem install` or the `/plugin` commands above.
### 🦞 OpenClaw Gateway
Install claude-mem as a persistent memory plugin on [OpenClaw](https://openclaw.ai) gateways with a single command:
```bash
curl -fsSL https://install.cmem.ai/openclaw.sh | bash
```
The installer handles dependencies, plugin setup, AI provider configuration, worker startup, and optional real-time observation feeds to Telegram, Discord, Slack, and more. See the [OpenClaw Integration Guide](https://docs.claude-mem.ai/openclaw-integration) for details.
**Key Features:**
- 🧠 **Persistent Memory** - Context survives across sessions
- 📊 **Progressive Disclosure** - Layered memory retrieval with token cost visibility
- 🔍 **Skill-Based Search** - Query your project history with mem-search skill
- 🖥️ **Web Viewer UI** - Real-time memory stream at http://localhost:37777
- 💻 **Claude Desktop Skill** - Search memory from Claude Desktop conversations
- 🔒 **Privacy Control** - Use `<private>` tags to exclude sensitive content from storage
- ⚙️ **Context Configuration** - Fine-grained control over what context gets injected
- 🤖 **Automatic Operation** - No manual intervention required
- 🔗 **Citations** - Reference past observations with IDs (access via http://localhost:37777/api/observation/{id} or view all in the web viewer at http://localhost:37777)
- 🧪 **Beta Channel** - Try experimental features like Endless Mode via version switching
---
## Documentation
📚 **[View Full Documentation](https://docs.claude-mem.ai/)** - Browse on official website
### Getting Started
- **[Installation Guide](https://docs.claude-mem.ai/installation)** - Quick start & advanced installation
- **[Gemini CLI Setup](https://docs.claude-mem.ai/gemini-cli/setup)** - Dedicated guide for Google's Gemini CLI integration
- **[Usage Guide](https://docs.claude-mem.ai/usage/getting-started)** - How Claude-Mem works automatically
- **[Search Tools](https://docs.claude-mem.ai/usage/search-tools)** - Query your project history with natural language
- **[Beta Features](https://docs.claude-mem.ai/beta-features)** - Try experimental features like Endless Mode
### Best Practices
- **[Context Engineering](https://docs.claude-mem.ai/context-engineering)** - AI agent context optimization principles
- **[Progressive Disclosure](https://docs.claude-mem.ai/progressive-disclosure)** - Philosophy behind Claude-Mem's context priming strategy
### Architecture
- **[Overview](https://docs.claude-mem.ai/architecture/overview)** - System components & data flow
- **[Architecture Evolution](https://docs.claude-mem.ai/architecture-evolution)** - The journey from v3 to v5
- **[Hooks Architecture](https://docs.claude-mem.ai/hooks-architecture)** - How Claude-Mem uses lifecycle hooks
- **[Hooks Reference](https://docs.claude-mem.ai/architecture/hooks)** - 7 hook scripts explained
- **[Worker Service](https://docs.claude-mem.ai/architecture/worker-service)** - HTTP API & Bun management
- **[Database](https://docs.claude-mem.ai/architecture/database)** - SQLite schema & FTS5 search
- **[Search Architecture](https://docs.claude-mem.ai/architecture/search-architecture)** - Hybrid search with Chroma vector database
### Configuration & Development
- **[Configuration](https://docs.claude-mem.ai/configuration)** - Environment variables & settings
- **[Development](https://docs.claude-mem.ai/development)** - Building, testing, contributing
- **[Troubleshooting](https://docs.claude-mem.ai/troubleshooting)** - Common issues & solutions
---
## How It Works
**Core Components:**
1. **5 Lifecycle Hooks** - SessionStart, UserPromptSubmit, PostToolUse, Stop, SessionEnd (6 hook scripts)
2. **Smart Install** - Cached dependency checker (pre-hook script, not a lifecycle hook)
3. **Worker Service** - HTTP API on port 37777 with web viewer UI and 10 search endpoints, managed by Bun
4. **SQLite Database** - Stores sessions, observations, summaries
5. **mem-search Skill** - Natural language queries with progressive disclosure
6. **Chroma Vector Database** - Hybrid semantic + keyword search for intelligent context retrieval
See [Architecture Overview](https://docs.claude-mem.ai/architecture/overview) for details.
---
## MCP Search Tools
Claude-Mem provides intelligent memory search through **4 MCP tools** following a token-efficient **3-layer workflow pattern**:
**The 3-Layer Workflow:**
1. **`search`** - Get compact index with IDs (~50-100 tokens/result)
2. **`timeline`** - Get chronological context around interesting results
3. **`get_observations`** - Fetch full details ONLY for filtered IDs (~500-1,000 tokens/result)
**How It Works:**
- Claude uses MCP tools to search your memory
- Start with `search` to get an index of results
- Use `timeline` to see what was happening around specific observations
- Use `get_observations` to fetch full details for relevant IDs
- **~10x token savings** by filtering before fetching details
**Available MCP Tools:**
1. **`search`** - Search memory index with full-text queries, filters by type/date/project
2. **`timeline`** - Get chronological context around a specific observation or query
3. **`get_observations`** - Fetch full observation details by IDs (always batch multiple IDs)
**Example Usage:**
```typescript
// Step 1: Search for index
search(query="authentication bug", type="bugfix", limit=10)
// Step 2: Review index, identify relevant IDs (e.g., #123, #456)
// Step 3: Fetch full details
get_observations(ids=[123, 456])
```
See [Search Tools Guide](https://docs.claude-mem.ai/usage/search-tools) for detailed examples.
---
## Beta Features
Claude-Mem offers a **beta channel** with experimental features like **Endless Mode** (biomimetic memory architecture for extended sessions). Switch between stable and beta versions from the web viewer UI at http://localhost:37777 → Settings.
See **[Beta Features Documentation](https://docs.claude-mem.ai/beta-features)** for details on Endless Mode and how to try it.
---
## System Requirements
- **Node.js**: 18.0.0 or higher
- **Claude Code**: Latest version with plugin support
- **Bun**: JavaScript runtime and process manager (auto-installed if missing)
- **uv**: Python package manager for vector search (auto-installed if missing)
- **SQLite 3**: For persistent storage (bundled)
---
### Windows Setup Notes
If you see an error like:
```powershell
npm : The term 'npm' is not recognized as the name of a cmdlet
```
Make sure Node.js and npm are installed and added to your PATH. Download the latest Node.js installer from https://nodejs.org and restart your terminal after installation.
---
## Configuration
Settings are managed in `~/.claude-mem/settings.json` (auto-created with defaults on first run). Configure AI model, worker port, data directory, log level, and context injection settings.
See the **[Configuration Guide](https://docs.claude-mem.ai/configuration)** for all available settings and examples.
### Mode & Language Configuration
Claude-Mem supports multiple workflow modes and languages via the `CLAUDE_MEM_MODE` setting.
This option controls both:
- The workflow behavior (e.g. code, chill, investigation)
- The language used in generated observations
#### How to Configure
Edit your settings file at `~/.claude-mem/settings.json`:
```json
{
"CLAUDE_MEM_MODE": "code--zh"
}
```
Modes are defined in `plugin/modes/`. To see all available modes locally:
```bash
ls ~/.claude/plugins/marketplaces/thedotmack/plugin/modes/
```
#### Available Modes
| Mode | Description |
|------------|-------------------------|
| `code` | Default English mode |
| `code--zh` | Simplified Chinese mode |
| `code--ja` | Japanese mode |
Language-specific modes follow the pattern `code--[lang]` where `[lang]` is the ISO 639-1 language code (e.g., `zh` for Chinese, `ja` for Japanese, `es` for Spanish).
> Note: `code--zh` (Simplified Chinese) is already built-in — no additional installation or plugin update is required.
#### After Changing Mode
Restart Claude Code to apply the new mode configuration.
---
## Development
See the **[Development Guide](https://docs.claude-mem.ai/development)** for build instructions, testing, and contribution workflow.
---
## Troubleshooting
If experiencing issues, describe the problem to Claude and the troubleshoot skill will automatically diagnose and provide fixes.
See the **[Troubleshooting Guide](https://docs.claude-mem.ai/troubleshooting)** for common issues and solutions.
---
## Bug Reports
Create comprehensive bug reports with the automated generator:
```bash
cd ~/.claude/plugins/marketplaces/thedotmack
npm run bug-report
```
## Contributing
Contributions are welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Make your changes with tests
4. Update documentation
5. Submit a Pull Request
See [Development Guide](https://docs.claude-mem.ai/development) for contribution workflow.
---
## License
Claude-Mem is licensed under the Apache License 2.0.
We chose Apache-2.0 because durable agentic memory should be easy to embed in
developer tools, local agents, MCP servers, enterprise systems, robotics stacks,
and production agent harnesses.
See the [LICENSE](LICENSE) file for full details. See [docs/license.md](docs/license.md)
and [docs/ip-boundary.md](docs/ip-boundary.md) for licensing scope and the
open/commercial boundary.
**Note on Ragtime**: The `ragtime/` directory is licensed under the **Apache License 2.0**. See [ragtime/LICENSE](ragtime/LICENSE) for details.
---
## Support
- **Documentation**: [docs/](docs/)
- **Issues**: [GitHub Issues](https://github.com/thedotmack/claude-mem/issues)
- **Repository**: [github.com/thedotmack/claude-mem](https://github.com/thedotmack/claude-mem)
- **Official X Account**: [@Claude_Memory](https://x.com/Claude_Memory)
- **Official Discord**: [Join Discord](https://discord.com/invite/J4wttp9vDu)
- **Author**: Alex Newman ([@thedotmack](https://github.com/thedotmack))
---
**Built with Claude Agent SDK** | **Works with Claude Code** | **Made with TypeScript**
---
### What About $CMEM?
$CMEM is a solana token created by a 3rd party without Claude-Mem's prior consent, but officially embraced by the creator of Claude-Mem (Alex Newman, @thedotmack). The token acts as a community catalyst for growth and a vehicle for bringing real-time agent data to the developers and knowledge workers that need it most. $CMEM: 2TsmuYUrsctE57VLckZBYEEzdokUF8j8e1GavekWBAGS