claude-mem/plans/2026-05-06-redis-dependency-strategy.md

Redis-Compatible Dependency Strategy for Claude-Mem

Date: 2026-05-06

Recommendation

Make BullMQ the queue engine, but do not treat Redis like a user-managed global service. Treat it like part of claude-mem's runtime.

Best fit for the "auto-install / it just works" product energy:

1. Prefer a claude-mem-owned local Redis-compatible sidecar process.
2. Prefer Valkey as the bundled/default local server where practical.
3. Accept an existing Redis/Valkey/Dragonfly URL when the user already has one.
4. Use package managers only as installers for the sidecar binary, not as long-running service managers.
5. Keep Windows as Docker/external-URL first unless we choose a supported native Redis-compatible build.

In settings and docs, call the capability redis-compatible queue store, but keep env names familiar:

CLAUDE_MEM_QUEUE_ENGINE=bullmq
CLAUDE_MEM_REDIS_MODE=managed|external|docker
CLAUDE_MEM_REDIS_URL=redis://127.0.0.1:<allocated-port>

Why Valkey-first for managed local mode

Valkey is a Redis-compatible fork under the Linux Foundation ecosystem, has current releases, Homebrew/package-manager install paths, Docker images, and Linux binary artifacts. It also gives claude-mem a cleaner dependency story for a managed local queue store.

Redis itself is still viable. Redis Open Source 8 has changed licensing over time, while Valkey keeps the local managed dependency straightforward for "we run a Redis-compatible queue store locally."

BullMQ's own docs say BullMQ is Redis-compliant with Redis 6.2+ but warns that not all Redis alternatives work properly. So this needs CI coverage. Dragonfly is officially called out by BullMQ as a supported/tested Redis-compatible alternative, but Dragonfly's own local install path is Docker-first, which is heavier than Valkey for claude-mem's installer.

Install decision tree

Interactive install

1. Probe for external config:

   • If CLAUDE_MEM_REDIS_URL exists, test PING, INFO, BullMQ Lua/script compatibility, and maxmemory-policy.
   • If valid, use it and do not manage the process.

2. Probe for local compatible binaries:

   • valkey-server
   • redis-server
   • known Homebrew paths: /opt/homebrew/bin, /usr/local/bin
   • Linux package paths: /usr/bin, /usr/local/bin

3. If a binary exists, create claude-mem's own config and data dir:

   • ~/.claude-mem/redis/redis.conf
   • ~/.claude-mem/redis/data/
   • ~/.claude-mem/redis/redis.pid
   • ~/.claude-mem/logs/redis-YYYY-MM-DD.log

4. If no binary exists:

   • macOS with Homebrew: install valkey with brew install valkey.
   • Linux with supported package manager: install valkey using apt/dnf/yum/apk/pacman when available.
   • Linux without package support but supported Ubuntu base: download Valkey binary artifact, verify SHA256, unpack under ~/.claude-mem/bin/valkey/<version>/.
   • Windows: use Docker if Docker is already present and running, otherwise ask for an external Redis URL or keep SQLite fallback.

5. Start the managed sidecar, then start the worker.

Non-interactive install

Default should not block on prompts:

• If CLAUDE_MEM_REDIS_URL works, use it.
• Else if a local valkey-server or redis-server exists, manage it.
• Else if --install-redis was passed, attempt platform install.
• Else fail with a precise command to run.

Do not surprise-run sudo apt install or install Docker in non-interactive mode.

Managed sidecar config

Use a private port, not global 6379.

Allocate and persist a queue-store port the same way claude-mem persists the worker port:

CLAUDE_MEM_REDIS_HOST=127.0.0.1
CLAUDE_MEM_REDIS_PORT=<free-port>
CLAUDE_MEM_REDIS_URL=redis://127.0.0.1:<free-port>

Suggested config:

bind 127.0.0.1 ::1
protected-mode yes
port <allocated-port>
dir ~/.claude-mem/redis/data
daemonize no
appendonly yes
appendfsync everysec
save 60 1
maxmemory-policy noeviction

BullMQ specifically requires maxmemory-policy=noeviction for correct queue behavior and recommends AOF persistence for production durability.

Do not use the user's global Redis config. Generate a claude-mem config so the queue store has the settings BullMQ needs.

Process model

Add RedisManager / QueueStoreManager alongside the worker supervisor:

• ensureQueueStoreStarted()
• stopQueueStore()
• queueStoreStatus()
• PID file with start-token validation, mirroring worker PID safety.
• Health probe:
  • TCP connect
  • PING
  • INFO server
  • CONFIG GET maxmemory-policy
  • CONFIG GET appendonly
  • BullMQ smoke queue add/get/remove in a namespaced key prefix

Worker startup sequence:

1. Load settings.
2. Ensure queue store is ready.
3. Initialize BullMQ connection.
4. Run SQLite migrations.
5. Start HTTP worker.

Shutdown sequence:

1. Stop providers/workers.
2. Close BullMQ connections.
3. Stop managed queue store only if claude-mem owns it.

Why not global service management

Avoid making the installer do this as the default:

• brew services start redis
• systemctl enable redis
• systemctl start valkey

Those mutate the user's machine globally, conflict with existing Redis installs, require sudo/admin flows, and make uninstall messy.

The better UX is a private local sidecar owned by claude-mem. It starts when claude-mem starts, stores data in ~/.claude-mem, and is removed by npx claude-mem uninstall.

Platform notes

macOS

Best path:

• If Homebrew exists: brew install valkey.
• Start valkey-server directly with claude-mem's generated config.
• Do not use brew services.

Redis official macOS install now uses brew tap redis/redis and brew install --cask redis, but Redis notes that this cask is not integrated with brew services. For claude-mem, that's fine because we should not rely on brew services anyway.

Linux

Best path:

• Prefer package-manager Valkey when available.
• On Ubuntu/Debian, Valkey docs list apt install valkey; Ubuntu also has valkey-redis-compat for redis-* symlinks.
• For Jammy/Noble, Valkey publishes binary artifacts, which are good candidates for a claude-mem-managed install under ~/.claude-mem/bin.

Windows

Hardest platform.

Redis official docs say Windows Redis Open Source requires Docker, with Memurai as a Windows compatibility partner. Valkey docs say Windows is not officially supported and suggest WSL for development.

Pragmatic options:

• If Docker is installed/running, launch valkey/valkey:<pinned> or redis:<pinned> with a named volume.
• If WSL is configured, install/run Valkey inside WSL and connect from Windows.
• Otherwise require CLAUDE_MEM_REDIS_URL or use temporary SQLite fallback until native Windows support is chosen.

Do not auto-install Docker Desktop. It is too invasive for an "it just works" CLI installer.

User-facing UX

Interactive:

Queue engine
  BullMQ needs a local Redis-compatible queue store.
  claude-mem can manage one for you under ~/.claude-mem.

  [recommended] Manage local Valkey for me
                Use existing Redis URL
                Keep SQLite queue for now

Non-interactive:

npx claude-mem install --queue bullmq --install-redis
npx claude-mem install --queue bullmq --redis-url redis://127.0.0.1:6379

Status:

npx claude-mem status

Worker:      running on 127.0.0.1:37777
Queue:       BullMQ
Store:       managed Valkey 9.0.3 on 127.0.0.1:39241
Persistence: AOF everysec
Policy:      noeviction

Uninstall:

• Stop managed queue store.
• Remove managed PID/config/logs as requested.
• Preserve queue data by default unless user passes --purge-data.

Implementation phases

1. Add queue-store settings and status plumbing.
2. Add QueueStoreManager with process spawn, PID validation, port allocation, and probes.
3. Add Valkey/Redis binary detection.
4. Add macOS/Linux install helpers.
5. Add BullMQ queue backend using managed store.
6. Add Windows Docker/external URL path.
7. Add uninstall cleanup.
8. Add CI matrix:
   • Redis 7.2 or Redis 8
   • Valkey 8/9
   • optional Dragonfly smoke test

Final call

For claude-mem's desired UX, the winning approach is:

BullMQ + claude-mem-managed Valkey sidecar by default, external Redis URL as an escape hatch, SQLite as short-term fallback only.

This gives the speed and correctness of Redis/BullMQ without making users become Redis operators.