Server-beta: Postgres storage + independent runtime + BullMQ queue (Phases 1–3) (#2351)
* Add server beta runtime foundation * Address server beta review findings * Resolve server beta review comments * Tighten server beta review follow-ups * Harden server beta auth and search * Avoid unnecessary FTS rebuilds * Block scoped keys from creating projects * Release BullMQ claims best effort on close * Address server beta review blockers * Reset BullMQ claims best effort * Add Postgres observation storage foundation * feat(server-beta): add independent runtime service Introduce src/server/runtime/ as a self-contained server-beta runtime that owns its lifecycle, Postgres bootstrap, and HTTP boundary without depending on WorkerService. ServerBetaService wraps the existing Server class, exposes /healthz and /v1/info with runtime="server-beta", and persists state to dedicated paths (.server-beta.pid|.port|.runtime.json). The four boundary managers (queue, generation worker, provider registry, event broadcaster) are intentionally disabled in this phase and report their status through /v1/info; later phases activate them. Adds plans/2026-05-07-finish-bullmq-branch-ship-plan.md to track the remaining work for this branch. Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): route CLI lifecycle and bundle separate runtime scripts/build-hooks.js now produces plugin/scripts/server-beta-service.cjs as a separate Node CJS bundle, alongside the existing worker-service bundle. The server-beta runtime is now installable independently. src/npx-cli/commands/server.ts routes start|stop|restart|status to the server-beta lifecycle instead of the legacy worker. The worker keeps its own start|stop|restart|status under the worker namespace; the two runtimes can be operated independently. src/services/worker-service.ts adds a server-* command parser branch that delegates to the sibling server-beta-service.cjs bundle so direct worker-service invocations still route to the right runtime. tests/npx-cli-server-namespace.test.ts updated to expect server-beta lifecycle routing. Includes rebuilt plugin/scripts/*.cjs bundles produced by build-and-sync. Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): add BullMQ job queue primitives Introduce src/server/jobs/ as the queue-side primitives that Phase 3 of the server-beta runtime needs to operate. types.ts defines a discriminated union over the four job kinds (event, event-batch, summary, reindex) and maps each to a per-kind BullMQ queue name and deterministic-ID prefix. job-id.ts builds deterministic, colon-free BullMQ jobIds from (kind, team, project, source). The colon ban exists because BullMQ uses ':' as a Redis key separator internally; embedding ':' in jobIds breaks scan and state lookups. ServerJobQueue.ts is a thin wrapper over BullMQ Queue + Worker that enforces autorun:false, default concurrency 1, and an attached error listener — all per BullMQ docs requirements. Test seams accept queue and worker factories so unit tests do not need Redis. outbox.ts publishes through the Postgres ObservationGenerationJob repository as canonical history. enqueueOutbox writes the row first, then publishes to BullMQ; if BullMQ throws, the row is transitioned to failed and a failed event is appended. reconcileOnStartup re-enqueues queued + processing rows after a restart, replacing terminal BullMQ jobs that may still be holding the deterministic ID slot. markCompleted and markFailed wrap transitionStatus and append the matching event row. Includes 20 unit tests covering deterministic ID stability, colon-free output, queue lifecycle, error-listener attachment, double-start refusal, idempotent enqueue, BullMQ failure rollback, startup reconciliation, max-attempts skipping, and completion / failure / retry transitions. Phase 3 commit 1 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): activate queue boundary in runtime service Wire ActiveServerBetaQueueManager into the server-beta runtime graph. The active manager owns one ServerJobQueue per generation kind (event, event-batch, summary, reindex) and surfaces lane metadata through boundary health. Selection is opt-in and fail-fast: if CLAUDE_MEM_QUEUE_ENGINE is set to bullmq the active manager is constructed (and any Redis/config error throws — no silent fallback to SQLite, per Phase 3 anti-pattern guard). For any other engine the disabled boundary remains so worker-era and test setups stay compatible. Widens ServerBetaBoundaryHealth.status to a discriminated union ('disabled' | 'active' | 'errored') with optional details. The disabled adapter still emits status='disabled', which keeps the existing server-beta-service test green. ServerBetaService receives the manager through a new optional queueManager field on CreateServerBetaServiceOptions so test graphs and Phase 4 wiring can inject custom managers. Adds tests/server/runtime/active-queue-manager.test.ts covering bullmq guard, active health shape, per-kind queue access, close behavior, and post-close errored health. Phase 3 commit 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): cap /v1/events/batch at 500 events Prevents unbounded array DoS surface flagged in PR review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alex Newman
@@ -41,14 +41,19 @@ USER root
|
||||
COPY plugin/ /opt/claude-mem/
|
||||
RUN chown -R node:node /opt/claude-mem
|
||||
|
||||
RUN mkdir -p /home/node/.claude /home/node/.claude-mem \
|
||||
&& chown -R node:node /home/node/.claude /home/node/.claude-mem
|
||||
|
||||
USER node
|
||||
WORKDIR /home/node
|
||||
RUN cd /opt/claude-mem \
|
||||
&& npm install --omit=dev --legacy-peer-deps
|
||||
|
||||
USER root
|
||||
RUN mkdir -p /home/node/.claude /home/node/.claude-mem /data/claude-mem \
|
||||
&& chown -R node:node /home/node/.claude /home/node/.claude-mem /data/claude-mem
|
||||
|
||||
COPY --chown=node:node docker/claude-mem/entrypoint.sh /usr/local/bin/claude-mem-entrypoint
|
||||
RUN chmod +x /usr/local/bin/claude-mem-entrypoint
|
||||
|
||||
USER node
|
||||
WORKDIR /home/node
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/claude-mem-entrypoint"]
|
||||
CMD ["bash"]
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
import net from 'node:net';
|
||||
|
||||
const baseUrl = process.env.E2E_BASE_URL ?? 'http://claude-mem-server:37777';
|
||||
const redisHost = process.env.E2E_REDIS_HOST ?? 'valkey';
|
||||
const redisPort = Number.parseInt(process.env.E2E_REDIS_PORT ?? '6379', 10);
|
||||
const phase = process.env.E2E_PHASE ?? 'phase1';
|
||||
const apiKey = requiredEnv('E2E_API_KEY');
|
||||
const readOnlyKey = process.env.E2E_READ_ONLY_API_KEY ?? '';
|
||||
const revokedKey = process.env.E2E_REVOKED_API_KEY ?? '';
|
||||
const runId = process.env.E2E_RUN_ID ?? `e2e-${Date.now()}`;
|
||||
const projectRoot = `/tmp/claude-mem-server-beta-${runId}`;
|
||||
|
||||
function requiredEnv(key) {
|
||||
const value = process.env[key];
|
||||
if (!value) {
|
||||
throw new Error(`${key} is required`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function assert(condition, message) {
|
||||
if (!condition) {
|
||||
throw new Error(message);
|
||||
}
|
||||
}
|
||||
|
||||
async function sleep(ms) {
|
||||
await new Promise(resolve => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
async function request(path, options = {}) {
|
||||
const headers = {
|
||||
...(options.json !== undefined ? { 'content-type': 'application/json' } : {}),
|
||||
...(options.apiKey ? { authorization: `Bearer ${options.apiKey}` } : {}),
|
||||
...(options.headers ?? {}),
|
||||
};
|
||||
return fetch(`${baseUrl}${path}`, {
|
||||
method: options.method ?? (options.json === undefined ? 'GET' : 'POST'),
|
||||
headers,
|
||||
body: options.json === undefined ? undefined : JSON.stringify(options.json),
|
||||
});
|
||||
}
|
||||
|
||||
async function json(response) {
|
||||
const text = await response.text();
|
||||
try {
|
||||
return text ? JSON.parse(text) : null;
|
||||
} catch (error) {
|
||||
throw new Error(`Invalid JSON response (${response.status}): ${text}\n${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function requestJson(path, options = {}) {
|
||||
const response = await request(path, options);
|
||||
const body = await json(response);
|
||||
return { response, body };
|
||||
}
|
||||
|
||||
async function expectStatus(path, status, options = {}) {
|
||||
const response = await request(path, options);
|
||||
assert(response.status === status, `${path} expected HTTP ${status}, got ${response.status}: ${await response.text()}`);
|
||||
}
|
||||
|
||||
async function waitForReadiness() {
|
||||
const deadline = Date.now() + 120_000;
|
||||
let lastError = '';
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
const health = await request('/healthz');
|
||||
const readiness = await request('/api/readiness');
|
||||
if (health.ok && readiness.ok) {
|
||||
return;
|
||||
}
|
||||
lastError = `health=${health.status} readiness=${readiness.status}`;
|
||||
} catch (error) {
|
||||
lastError = error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
await sleep(1000);
|
||||
}
|
||||
throw new Error(`Server did not become ready: ${lastError}`);
|
||||
}
|
||||
|
||||
async function assertRedisPing() {
|
||||
const result = await new Promise((resolve, reject) => {
|
||||
const socket = net.createConnection({ host: redisHost, port: redisPort });
|
||||
socket.setTimeout(3000);
|
||||
let data = '';
|
||||
socket.on('connect', () => socket.write('*1\r\n$4\r\nPING\r\n'));
|
||||
socket.on('data', chunk => {
|
||||
data += chunk.toString('utf8');
|
||||
if (data.includes('PONG')) {
|
||||
socket.end();
|
||||
resolve(data);
|
||||
}
|
||||
});
|
||||
socket.on('timeout', () => {
|
||||
socket.destroy();
|
||||
reject(new Error('Redis PING timed out'));
|
||||
});
|
||||
socket.on('error', reject);
|
||||
socket.on('close', () => {
|
||||
if (!data.includes('PONG')) {
|
||||
reject(new Error(`Redis PING failed: ${data}`));
|
||||
}
|
||||
});
|
||||
});
|
||||
assert(String(result).includes('PONG'), `Redis did not return PONG: ${result}`);
|
||||
}
|
||||
|
||||
async function assertQueueHealth() {
|
||||
const { response, body } = await requestJson('/api/health');
|
||||
assert(response.ok, `/api/health expected OK, got ${response.status}`);
|
||||
assert(body.queue?.engine === 'bullmq', `expected BullMQ queue engine, got ${JSON.stringify(body.queue)}`);
|
||||
assert(body.queue?.redis?.status === 'ok', `expected Redis health ok, got ${JSON.stringify(body.queue?.redis)}`);
|
||||
assert(body.queue?.redis?.mode === 'docker', `expected docker Redis mode, got ${JSON.stringify(body.queue?.redis)}`);
|
||||
}
|
||||
|
||||
async function phase1() {
|
||||
console.log(`[e2e] phase1 starting (${runId})`);
|
||||
await waitForReadiness();
|
||||
await assertQueueHealth();
|
||||
await assertRedisPing();
|
||||
|
||||
await expectStatus('/v1/projects', 401, {
|
||||
method: 'POST',
|
||||
json: { name: 'unauthenticated' },
|
||||
});
|
||||
await expectStatus('/v1/projects', 403, {
|
||||
method: 'POST',
|
||||
apiKey: 'cmem_invalid_key',
|
||||
json: { name: 'invalid' },
|
||||
});
|
||||
if (readOnlyKey) {
|
||||
await expectStatus('/v1/projects', 403, {
|
||||
method: 'POST',
|
||||
apiKey: readOnlyKey,
|
||||
json: { name: 'read-only denied' },
|
||||
});
|
||||
const readOnlyProjects = await request('/v1/projects', { apiKey: readOnlyKey });
|
||||
assert(readOnlyProjects.ok, `read-only key should read projects, got ${readOnlyProjects.status}`);
|
||||
}
|
||||
|
||||
const createdProject = await requestJson('/v1/projects', {
|
||||
apiKey,
|
||||
json: {
|
||||
name: `Server Beta E2E ${runId}`,
|
||||
rootPath: projectRoot,
|
||||
metadata: { runId },
|
||||
},
|
||||
});
|
||||
assert(createdProject.response.status === 201, `project create failed: ${JSON.stringify(createdProject.body)}`);
|
||||
const project = createdProject.body.project;
|
||||
assert(project?.id, 'project response missing id');
|
||||
|
||||
const createdSession = await requestJson('/v1/sessions/start', {
|
||||
apiKey,
|
||||
json: {
|
||||
projectId: project.id,
|
||||
contentSessionId: `content-${runId}`,
|
||||
memorySessionId: `memory-${runId}`,
|
||||
platformSource: 'docker-e2e',
|
||||
title: 'Docker E2E session',
|
||||
},
|
||||
});
|
||||
assert(createdSession.response.status === 201, `session create failed: ${JSON.stringify(createdSession.body)}`);
|
||||
const session = createdSession.body.session;
|
||||
|
||||
const createdEvent = await requestJson('/v1/events', {
|
||||
apiKey,
|
||||
json: {
|
||||
projectId: project.id,
|
||||
serverSessionId: session.id,
|
||||
sourceType: 'api',
|
||||
eventType: 'observation.created',
|
||||
contentSessionId: `content-${runId}`,
|
||||
memorySessionId: `memory-${runId}`,
|
||||
payload: { tool_name: 'Read', runId },
|
||||
occurredAtEpoch: Date.now(),
|
||||
},
|
||||
});
|
||||
assert(createdEvent.response.status === 201, `event create failed: ${JSON.stringify(createdEvent.body)}`);
|
||||
const event = createdEvent.body.event;
|
||||
|
||||
const batchEvents = await requestJson('/v1/events/batch', {
|
||||
apiKey,
|
||||
json: [
|
||||
{
|
||||
projectId: project.id,
|
||||
sourceType: 'api',
|
||||
eventType: 'observation.created',
|
||||
payload: { index: 1, runId },
|
||||
occurredAtEpoch: Date.now(),
|
||||
},
|
||||
{
|
||||
projectId: project.id,
|
||||
sourceType: 'api',
|
||||
eventType: 'observation.created',
|
||||
payload: { index: 2, runId },
|
||||
occurredAtEpoch: Date.now(),
|
||||
},
|
||||
],
|
||||
});
|
||||
assert(batchEvents.response.status === 201, `event batch failed: ${JSON.stringify(batchEvents.body)}`);
|
||||
assert(batchEvents.body.events.length === 2, 'event batch did not return two events');
|
||||
|
||||
const fetchedEvent = await requestJson(`/v1/events/${event.id}`, { apiKey });
|
||||
assert(fetchedEvent.response.ok, `event fetch failed: ${JSON.stringify(fetchedEvent.body)}`);
|
||||
|
||||
const createdMemory = await requestJson('/v1/memories', {
|
||||
apiKey,
|
||||
json: {
|
||||
projectId: project.id,
|
||||
serverSessionId: session.id,
|
||||
kind: 'manual',
|
||||
type: 'decision',
|
||||
title: `Docker E2E memory ${runId}`,
|
||||
narrative: `Server beta Docker E2E memory survives restart for ${runId}.`,
|
||||
facts: ['BullMQ health is backed by Valkey', `run:${runId}`],
|
||||
concepts: ['server-beta', 'docker-e2e'],
|
||||
metadata: { runId },
|
||||
},
|
||||
});
|
||||
assert(createdMemory.response.status === 201, `memory create failed: ${JSON.stringify(createdMemory.body)}`);
|
||||
const memory = createdMemory.body.memory;
|
||||
|
||||
const patchedMemory = await requestJson(`/v1/memories/${memory.id}`, {
|
||||
method: 'PATCH',
|
||||
apiKey,
|
||||
json: {
|
||||
projectId: project.id,
|
||||
kind: 'manual',
|
||||
type: 'decision',
|
||||
narrative: `Patched Docker E2E memory survives restart for ${runId}.`,
|
||||
facts: ['patched', `run:${runId}`],
|
||||
},
|
||||
});
|
||||
assert(patchedMemory.response.ok, `memory patch failed: ${JSON.stringify(patchedMemory.body)}`);
|
||||
assert(patchedMemory.body.memory.narrative.includes('Patched'), 'patched memory narrative was not returned');
|
||||
|
||||
const fetchedMemory = await requestJson(`/v1/memories/${memory.id}`, { apiKey });
|
||||
assert(fetchedMemory.response.ok, `memory fetch failed: ${JSON.stringify(fetchedMemory.body)}`);
|
||||
|
||||
const search = await requestJson('/v1/search', {
|
||||
apiKey,
|
||||
json: { projectId: project.id, query: runId, limit: 10 },
|
||||
});
|
||||
assert(search.response.ok, `search failed: ${JSON.stringify(search.body)}`);
|
||||
assert(search.body.memories.some(item => item.id === memory.id), 'search did not return created memory');
|
||||
|
||||
const context = await requestJson('/v1/context', {
|
||||
apiKey,
|
||||
json: { projectId: project.id, query: 'patched', limit: 5 },
|
||||
});
|
||||
assert(context.response.ok, `context failed: ${JSON.stringify(context.body)}`);
|
||||
assert(context.body.context.includes(runId), 'context did not include created memory text');
|
||||
|
||||
const endedSession = await requestJson(`/v1/sessions/${session.id}/end`, {
|
||||
method: 'POST',
|
||||
apiKey,
|
||||
json: {},
|
||||
});
|
||||
assert(endedSession.response.ok, `session end failed: ${JSON.stringify(endedSession.body)}`);
|
||||
assert(endedSession.body.session.status === 'completed', 'session did not complete');
|
||||
|
||||
const audit = await requestJson(`/v1/audit?projectId=${encodeURIComponent(project.id)}`, { apiKey });
|
||||
assert(audit.response.ok, `audit failed: ${JSON.stringify(audit.body)}`);
|
||||
assert(audit.body.audit.some(row => row.action === 'memory.write'), 'audit log missing memory.write');
|
||||
|
||||
console.log(`[e2e] phase1 passed project=${project.id} memory=${memory.id}`);
|
||||
}
|
||||
|
||||
async function phase2() {
|
||||
console.log(`[e2e] phase2 after restart starting (${runId})`);
|
||||
await waitForReadiness();
|
||||
await assertQueueHealth();
|
||||
await assertRedisPing();
|
||||
|
||||
if (revokedKey) {
|
||||
await expectStatus('/v1/projects', 403, { apiKey: revokedKey });
|
||||
}
|
||||
|
||||
const projects = await requestJson('/v1/projects', { apiKey });
|
||||
assert(projects.response.ok, `project list failed after restart: ${JSON.stringify(projects.body)}`);
|
||||
const project = projects.body.projects.find(item => item.rootPath === projectRoot);
|
||||
assert(project?.id, `persisted project not found for ${projectRoot}`);
|
||||
|
||||
const search = await requestJson('/v1/search', {
|
||||
apiKey,
|
||||
json: { projectId: project.id, query: runId, limit: 10 },
|
||||
});
|
||||
assert(search.response.ok, `search failed after restart: ${JSON.stringify(search.body)}`);
|
||||
assert(search.body.memories.some(item => String(item.narrative ?? '').includes(runId)), 'persisted memory not found after restart');
|
||||
|
||||
const audit = await requestJson(`/v1/audit?projectId=${encodeURIComponent(project.id)}`, { apiKey });
|
||||
assert(audit.response.ok, `audit failed after restart: ${JSON.stringify(audit.body)}`);
|
||||
assert(audit.body.audit.length > 0, 'audit log did not persist after restart');
|
||||
|
||||
console.log(`[e2e] phase2 passed project=${project.id}`);
|
||||
}
|
||||
|
||||
if (phase === 'phase1') {
|
||||
await phase1();
|
||||
} else if (phase === 'phase2') {
|
||||
await phase2();
|
||||
} else {
|
||||
throw new Error(`Unknown E2E_PHASE: ${phase}`);
|
||||
}
|
||||
Reference in New Issue
Block a user