server-beta: Phases 4–13 — event pipeline, generation, MCP, compat, Docker, team audit, observability (#2383)
* feat(server-beta): Phase 4 — Postgres event-to-generation-job pipeline Adds POST /v1/events, /v1/events/batch, GET /v1/jobs/:id, GET /v1/events/:id, and POST /v1/memories on the server-beta runtime, backed by Postgres. - Event row + outbox generation-job row insert in one withPostgresTransaction. - BullMQ enqueue happens after commit; enqueue failure leaves the row queued for Phase 3 startup reconciliation. - ?generate=false skips the outbox; ?wait=true returns queue status only, never observation IDs (provider generation is Phase 5). - Batch pre-validates all event projectIds against api-key scope before any write; mixed-project batches reject 403 with zero side effects. - /v1/memories is a direct insert alias — no generator, no outbox. - Cross-tenant /v1/jobs/:id returns 404 to avoid leaking row existence. - New PostgresAuthMiddleware reads api_keys by SHA-256 hash; populates req.authContext.teamId/projectId; legacy ServerV1Routes (SQLite, used by worker runtime) is left untouched. - Tests: unit suite hardened with stubbed pool.query so route registration is safe; integration tests skip cleanly without CLAUDE_MEM_TEST_POSTGRES_URL. Verification: 87 pass / 1 skip / 0 fail. No new typecheck errors. Required greps for WorkerService and MemoryItemsRepository in src/server/routes/v1 and src/server/runtime return no hits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 5 — provider observation generator Adds independent provider generation under src/server/generation/ with no worker coupling. Server beta can now generate observations end-to-end: event -> outbox -> BullMQ -> provider -> parser -> persisted observation. - ProviderObservationGenerator orchestrates: lock outbox (queued -> processing), reload agent_event from Postgres (BullMQ payload is advisory only), call provider, hand raw text to processGeneratedResponse, route errors via markGenerationFailed with retryable flag from ServerClassifiedProviderError. - processGeneratedResponse parses with parseAgentXml, persists via PostgresObservationRepository with deterministic generation_key = generation:v1:{job_id}:{index}:{fingerprint}, links via PostgresObservationSourcesRepository, advances outbox status, appends observation_generation_job_events, audits — all in one withPostgresTransaction. Idempotent on retry via UNIQUE constraints. - Three provider adapters under src/server/generation/providers/: Claude, Gemini, OpenRouter. Self-contained — no imports from src/services/worker/*. Worker providers unchanged. - Shared error classification + prompt builder under providers/shared/. Prompt builder strips <private> at the edge; fully-private batches emit <skip_summary /> without billing the provider. - ActiveServerBetaGenerationWorkerManager wires BullMQ Worker via ServerJobQueue.start(...) with concurrency 1 + autorun:false + worker.on('error') per BullMQ docs. - New GET /v1/events/:id/observations on ServerV1PostgresRoutes returns observations linked via observation_sources, team/project scoped. Verification: 104 pass / 4 skip / 0 fail. No typecheck regressions. Anti-pattern greps clean for services/worker imports under src/server, WorkerRef/ActiveSession/SessionStore in src/server/generation. Deferred: ModeManager loading uses a stable fallback observation type list; summary and reindex queue lanes are not yet wired. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 6 — independent server session semantics server_sessions is now the canonical Server beta session model. Sessions are independent of legacy worker ActiveSession state. - PostgresServerSessionRepository extended: findByExternalIdForScope, endSession (idempotent via COALESCE(ended_at, now())), markGenerationStarted/Completed/Failed, listUnprocessedEvents (filters agent_events with completed agent_event jobs). - ServerSessionRuntimeRepository wraps the repo; every method requires explicit team_id + project_id and validates scope via assertProjectOwnership. - SessionGenerationPolicy supports per-event (default), debounce (BullMQ delayed-job replace via getJob+remove+add), and end-of-session. Configured via CLAUDE_MEM_SERVER_SESSION_POLICY and CLAUDE_MEM_SERVER_SESSION_DEBOUNCE_MS env vars; per-team override hooks are exposed on ServerV1PostgresRoutesOptions for future settings layer. - POST /v1/sessions/start (find-or-create on (project_id, external_session_id), GET /v1/sessions/:id (scoped 404), POST /v1/sessions/:id/end (transactional: end + create summary outbox via UNIQUE collapse + enqueue post-commit). Re-ending is fully idempotent. - processSessionSummaryResponse persists summary as kind='summary' observation with the same idempotency model (generation_key + observation_sources UNIQUE). - ProviderObservationGenerator dispatches on source_type: agent_event -> processGeneratedResponse, session_summary -> processSessionSummaryResponse; loadEvents handles session-summary by loading unprocessed events. - ActiveServerBetaGenerationWorkerManager wires summary BullMQ lane alongside event lane (concurrency=1, autorun=false, error listener attached per BullMQ docs). Verification: 110 pass / 6 skip / 0 fail. Net typecheck error count unchanged at 24 (pre-existing, none in Phase 6 files). Anti-pattern greps clean for ActiveSession/SessionStore in src/server/runtime, no worker imports anywhere in src/server. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 7 — hook routing without worker dependency Hooks can now talk directly to server-beta when CLAUDE_MEM_RUNTIME=server-beta is selected, with a clean worker fallback when server-beta is unhealthy. - src/services/hooks/server-beta-client.ts — typed HTTP client for /v1/sessions/start, /v1/events, /v1/sessions/:id/end. Throws ServerBetaClientError with kind classification (missing_api_key, transport, timeout, http_error, invalid_response) and isFallbackEligible helper. Zero imports from services/worker/. - src/services/hooks/runtime-selector.ts — reads CLAUDE_MEM_RUNTIME from settings, returns worker or server-beta context, logs [server-beta-fallback] reason=<code> on every config-time fallback. - src/services/hooks/server-beta-bootstrap.ts — Postgres-backed API key bootstrap. Find-or-creates local-hook-team + local-hook-project, generates cmem_<random> key (SHA-256 hashed), inserts into api_keys with scopes events:write/sessions:write/observations:read/jobs:read. Settings file written with chmod 0600. rotateServerBetaApiKey() wired to a new `claude-mem server keys rotate` command. - src/cli/handlers/{observation,session-init,summarize}.ts — every hook handler tries server-beta first when configured, falls through to the existing worker path on transport/5xx/429/missing-key. One WARN line per fallback. Hook JSON output shape unchanged. - src/shared/SettingsDefaultsManager.ts — three new keys with defaults: CLAUDE_MEM_SERVER_BETA_URL, CLAUDE_MEM_SERVER_BETA_API_KEY, CLAUDE_MEM_SERVER_BETA_PROJECT_ID. - src/npx-cli/commands/install.ts — when installer selects server-beta runtime and CLAUDE_MEM_SERVER_DATABASE_URL is set, bootstraps a local API key automatically. Warns and continues if the DB URL is missing. plugin/scripts/*.cjs bundles rebuilt via npm run build to pick up the new hook handler code path. No plaintext keys in the bundle (verified). Verification: 16 hook unit tests pass; 275 server/storage/services tests pass with 7 pre-existing failures (verified independent of this change via git stash --include-untracked). Build clean. No new typecheck errors in Phase 7 files. Anti-pattern guards verified: - /api/sessions/observations only reached via explicit fallback path - server-beta runtime never starts the worker process - API keys live only in ~/.claude-mem/settings.json (chmod 0600), never in the bundle (grep confirmed) - Worker fallback preserved, observable via single WARN line per call Deferred: semantic context injection (UserPromptSubmit hook) stays worker-only; server-beta does not yet expose /v1/context/semantic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 8 — MCP backed by server-beta core MCP tools now route through server-beta in server-beta mode while keeping worker-mode search/timeline/get_observations tools fully working. - src/servers/mcp-server.ts — five new observation_* tools registered: observation_add, observation_record_event, observation_search, observation_context, observation_generation_status. Three memory_* compatibility aliases delegate to the canonical handlers. Worker auto-start is gated when selectRuntime() === 'server-beta' so MCP in server-beta mode never spawns the worker. - src/services/hooks/server-beta-client.ts — addObservation, searchObservations, contextObservations, getJobStatus added so MCP shares one transport with hooks (Phase 7). - src/server/routes/v1/ServerV1PostgresRoutes.ts — POST /v1/search and POST /v1/context REST cores backed by PostgresObservationRepository full-text search (GIN tsvector from Phase 1). - Existing memory_search/timeline/get_observations tools call callWorkerAPI unchanged in worker mode; worker tests unaffected. Verification: 39 pass / 4 skip / 0 fail on targeted suite. Pre-existing 7 baseline failures verified independent (git stash). No new typecheck errors. WorkerService grep clean across src/servers/mcp-server.ts and src/server/. Anti-pattern guards verified: - No duplicate generation logic in MCP — observation_record_event hits /v1/events which owns event+outbox+enqueue inside one tx - WorkerService not imported anywhere under MCP server-beta path - No hardcoded worker URLs — all transport via Phase 7 ServerBetaClient - memory_* aliases retained, single handler per pair Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 9 — compatibility adapters without coupling Legacy /api/sessions/observations and /api/sessions/summarize endpoints keep working on server-beta runtime by translating to AgentEvent and session-end calls — no worker code, no route duplication. - src/server/services/IngestEventsService.ts — shared event-ingest path used by both /v1/events and the compat adapter. Owns transactional event row + outbox row + lifecycle log + post-commit BullMQ enqueue, honors Phase 6 SessionGenerationPolicy. - src/server/services/EndSessionService.ts — shared session-end path used by both /v1/sessions/:id/end and the compat adapter. Idempotent ended_at + summary outbox + deterministic summary job id. - src/server/compat/SessionsObservationsAdapter.ts — translates legacy POST /api/sessions/observations payload (Claude Code transcript shape) -> AgentEvent (source_adapter='claude-code-compat', event_type='tool_use') -> IngestEventsService.ingestOne. Resolves contentSessionId to server_sessions via find-or-create. - src/server/compat/SessionsSummarizeAdapter.ts — translates legacy POST /api/sessions/summarize -> EndSessionService.end. Preserves the legacy agentId -> {status:'skipped', reason:'subagent_context'} behavior so existing clients see the same response shape. - src/server/routes/v1/ServerV1PostgresRoutes.ts — refactored to delegate to the new shared services (-203 LoC net) so /v1 and /api compat both call the SAME canonical code path. - src/server/runtime/ServerBetaService.ts — registers both compat adapters alongside ServerV1PostgresRoutes, sharing service instances. - docs/server-beta-parity-map.md — full enumeration of legacy /api/* routes labeled native, adapter, or unsupported (with reasons). Viewer read-path adapters explicitly listed as unsupported pending a future viewer-rewrite phase. Verification: 7 compat tests pass, 6 v1-routes tests still pass (refactor preserved behavior), 4 session-routes tests pass. Pre- existing 16 baseline failures verified independent via git stash. Zero new typecheck errors. Anti-pattern guards verified: - No services/worker/http/routes or WorkerService imports under src/server/compat or src/server/runtime - Compat adapters are thin translators with names ending in *Adapter and a top-of-file comment noting they are legacy compatibility - /v1/* remains the canonical Server beta API; compat adapters call shared services rather than acting as a parallel API Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 10 — Docker stack and deployable runtime Server beta now ships as a Docker stack with no worker process anywhere and a separate horizontal generation worker for scaling. - src/server/runtime/create-server-beta-service.ts — validateServerBetaEnv() fails fast on missing CLAUDE_MEM_SERVER_DATABASE_URL, requires CLAUDE_MEM_QUEUE_ENGINE=bullmq in Docker, rejects CLAUDE_MEM_AUTH_MODE=local-dev and CLAUDE_MEM_ALLOW_LOCAL_DEV_BYPASS inside containers (detected via /.dockerenv or CLAUDE_MEM_DOCKER=1). Adds CLAUDE_MEM_GENERATION_DISABLED so the HTTP service can run generator-free. - src/server/runtime/ServerBetaService.ts — runServerBetaGenerationWorker for the dedicated consumer process; runServerBetaApiKeyCli is a new Postgres-backed `server api-key` command (the legacy worker CLI wrote to SQLite and was invisible to the Postgres runtime); getQueueHealth shim feeds /api/health a consistent ObservationQueueHealth shape. - src/npx-cli/commands/{runtime,server}.ts — `claude-mem server worker start` subcommand that boots only the BullMQ consumer. - docker/claude-mem/{Dockerfile,entrypoint.sh} — entrypoint forces CLAUDE_MEM_DOCKER=1 + CLAUDE_MEM_RUNTIME=server-beta and exposes three modes: server (HTTP only, generation disabled), worker (BullMQ consumer), shell. Worker bundle is no longer the default CMD. - docker-compose.yml — full stack: postgres + valkey + claude-mem-server (HTTP-only) + claude-mem-worker (generation consumer). Wires service-to-service env vars. - scripts/e2e-server-beta-docker.sh + docker/e2e/server-beta-e2e.mjs — E2E now hits /v1/sessions/start, /v1/events?wait=true, /v1/jobs/:id; asserts no worker-service.cjs process anywhere in the stack; one-shot docker compose run --rm verifies local-dev auth is rejected with the expected stderr; restart-and-verify confirms Postgres durability and BullMQ retry idempotency. - docs/server.md — full Phase 10 doc: stack diagram, env table, worker mode, auth-in-Docker policy. - docs/api.md — event generation semantics (wait=true, generationJob). Verification: full Docker E2E PASSED on live daemon (phase1 + phase2 + restart-and-verify + revoked-key + no-worker- process + local-dev-rejected). Unit tests 292 pass / 9 skip / 7 fail (7 fails pre-existing baseline). Zero new typecheck errors. Anti-pattern guards verified: - entrypoint never execs worker-service.cjs; E2E greps prove no worker process anywhere in the stack - validateServerBetaEnv refuses local-dev auth in Docker with explicit remediation message; ALLOW_LOCAL_DEV_BYPASS rejected the same way - Docker requires CLAUDE_MEM_QUEUE_ENGINE=bullmq; in-process queue rejected at startup - claude-mem worker / worker-service / WorkerService greps clean in docker/ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 11 — team-aware generation with audit chain Generation jobs now carry team_id/project_id/api_key_id/actor_id/ source_adapter from enqueue through execution; the outbox is reloaded from Postgres before any side effect so BullMQ payload can never act as auth authority. - src/server/jobs/types.ts — ServerGenerationJobPayloadSchema (Zod discriminated union) requires team_id, project_id, generation_job_id, source_adapter, api_key_id, actor_id (nullable), source_type, source_id, plus event_id / server_session_id per kind. assertServerGenerationJobPayload is called at enqueue (outbox.ts) and again at execution boundary. - src/server/services/{IngestEventsService,EndSessionService}.ts + SessionGenerationPolicy.ts — thread identity context (apiKeyId, actorId, sourceAdapter) into both event and summary BullMQ payloads. - src/server/generation/ProviderObservationGenerator.ts — loadCanonicalOutbox loads the outbox row WITHOUT scope filter, then compares candidate.team_id/project_id to payload.team_id/project_id; mismatch -> ServerGenerationScopeViolationError (non-retryable), failed status, generation_job.scope_violation audit. isApiKeyRevoked checks api_keys (revoked_at, expires_at, row missing) before any provider call; revoked -> generation_job.revoked_key audit + non- retryable failure. generation_job.processing audit emitted on lock. - src/server/generation/processGeneratedResponse.ts — generated observations carry team_id/project_id/server_session_id from the reloaded source row (not job payload). observation_sources.metadata records source_adapter, actor_id, api_key_id for traceability. observation.created audit per observation; generation_job.completed audit per terminal transition. All audit rows reference the same generation_job_id in details. - src/server/routes/v1/ServerV1PostgresRoutes.ts — GET /v1/teams/:id/jobs and GET /v1/projects/:id/jobs with SQL-layer scoping (WHERE team_id=$1 [AND project_id=$2] [AND status=$3]); cross-tenant returns 404 to avoid leaking row existence. Pagination via status/limit/offset. audit_log rows for event.received, event.batch_received, observation.read. - src/server/compat/{SessionsObservationsAdapter,SessionsSummarizeAdapter}.ts — propagate apiKeyId and sourceAdapter='claude-code-compat'. Verification: 162 pass / 10 skip / 0 fail. Pre-existing failures in tests/services/queue and tests/services/worker confirmed independent via git stash. Zero new typecheck errors in server-beta files. Required greps: rg "team_id.*req\.body|project_id.*req\.body" src/server -> 0 matches Audit chain integration test passes — generation_job.processing, observation.created, and generation_job.completed audit rows all share the same generation_job_id reference. Anti-pattern guards verified: - BullMQ payload never acts as auth authority — Postgres outbox reload with mismatch check happens before every side effect - team_id / project_id never derived from request body for scope decisions; always req.authContext.teamId / projectId - Application-layer team/project filtering forbidden — listJobsForScope pushes scope into the SQL WHERE clause - Project-scoped key on cross-project /v1/teams/:id/jobs returns 404 - Revoked api keys cause non-retryable failure with audit before any provider call Deferred: a redundant generation_job.queued audit_log row (already covered by observation_generation_job_events lifecycle log per Phase 1 schema split). Compat adapters set actor_id=null but propagate api_key_id which is the canonical reference downstream. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 12 — observability and operations Operators can now inspect, retry, and cancel generation jobs from the CLI; queue lane metrics flow into /api/health and /v1/info; every request gets a stable request_id that flows through HTTP -> audit -> outbox -> generator -> completion log. - src/server/middleware/request-id.ts — honors safe inbound X-Request-Id, mints uuid v4 otherwise. Set on req.requestId and echoed via response header so external traces can correlate. - src/server/jobs/ServerJobQueue.ts — QueueEvents wired with completed, failed, progress, stalled, error listeners; lifecycle counters exposed via observe() API. Logs emitted as [generation] job=<id> source_type=<...> duration=<ms> attempts=<N> reason=<message>. Stalled and error counters survive worker restart. - src/server/jobs/types.ts — ServerGenerationJob payload schema extended with optional request_id; flows through from HTTP into every BullMQ job. - src/server/queue/ObservationQueueEngine.ts — health snapshot now carries per-lane (event, summary) counts via ObservationQueueHealthLaneSnapshot. - src/server/runtime/{ActiveServerBetaQueueManager, ActiveServerBetaGenerationWorkerManager,ServerBetaService}.ts — per-lane getJobCounts feed /api/health and /v1/info; stalled events audit through audit_log with action generation_job.stalled. - src/server/routes/v1/ServerV1PostgresRoutes.ts — GET /v1/jobs (status/source_type/since/limit/offset, scope from api-key, payload stripped unless ?include=payload AND admin scope), POST /v1/jobs/:id/retry (idempotent; queued -> no-op; audit generation_job.retried_by_operator), POST /v1/jobs/:id/cancel (terminal -> no-op; audit generation_job.cancelled_by_operator; generator reload-before-side-effects already prevents double work). - src/server/services/IngestEventsService.ts + SessionGenerationPolicy.ts + ProviderObservationGenerator.ts — request_id propagated end to end. Generator extracts request_id from BullMQ payload and includes it in lock/processing/completion logs and audit details. - src/npx-cli/commands/server-jobs.ts + src/npx-cli/commands/server.ts — `claude-mem server jobs status|failed|retry|cancel`. status compares Postgres outbox counts to BullMQ queue counts and surfaces divergence. failed prints attempts + last_error message. --team and --project filters. Verification: 350 pass / 12 skip / 7 fail (pre-existing baseline, verified independent via git stash). 18 new tests added (request-id middleware, server-jobs CLI seams, jobs list/retry/cancel routes Postgres-gated). Zero new typecheck errors. Anti-pattern guards verified: - agent_events.payload only emitted in /v1/jobs response inside the admin-gated branch (?include=payload + admin scope) — returns 403 otherwise - jobs retry on a queued row is a no-op (no double BullMQ enqueue, no double UPDATE) - Every operator action writes to audit_log with the *_by_operator action and request_id correlation in details - Stalled events audit through generation_job.stalled Sample correlated trace (one request_id end to end): HTTP middleware: req.requestId = 'req-abc' audit event.received: details.requestId = 'req-abc' BullMQ payload: { request_id: 'req-abc', generation_job_id: 'gj_x' } generator lock log: [generation] job locked { jobId, requestId } audit generation_job.processing: details.requestId = 'req-abc' completion log: [generation] job=evt_... duration=1230ms Deferred: live /api/health round-trip integration test (needs Redis); stalled event live integration test (needs Redis); storing request_id on the observations row itself (spec did not require). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(server-beta): add Phase 13 release readiness report Captures the final verification gate: tests (1749 pass, 45 fail all pre-existing baseline, zero regressions), required greps clean, Docker E2E green end-to-end, all 7 exit criteria met, build clean, typecheck unchanged from main. Documents deferred items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(server-beta): rebuild server-beta-service bundle Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address Greptile review on PR #2383 - ProviderObservationGenerator.lockOutbox: skip duplicate worker run when another lock is active instead of returning the row, which previously let two BullMQ workers issue the (paid, rate-limited) external provider call before the persistence-layer terminal-status guard collapsed the duplicate. Reconciliation still recovers from a stale lock on startup or next retry. - docker-compose.yml: require POSTGRES_USER/PASSWORD/DB env vars (no defaults). Stack refuses to start without explicit secrets. Added a header warning that the file must not be deployed unmodified. - e2e-server-beta-docker.sh: export ephemeral test creds for the new required env vars so the Docker E2E driver still runs unattended. - ServerBetaService api-key list: bound query with LIMIT/OFFSET (default 100, max 500) and add optional --team filter to prevent unintentional cross-tenant key metadata disclosure on shared admin hosts. - SessionGenerationPolicy: fix dead `??` fallback for NaN parseInt result; use `||` so DEFAULT_DEBOUNCE_MS actually applies. - ServerV1PostgresRoutes: `?wait=true` now actually waits — polls the outbox row until terminal status (timeout 30s, 100ms interval) on both /v1/events and /v1/events/batch. Returns `waitTimedOut: true` if the cap is hit so callers can re-poll the status endpoints. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address CodeRabbit + Greptile second review on PR #2383 P1 fixes - Operator retry endpoint was re-publishing the Postgres outbox metadata column as the BullMQ payload; the worker's assertServerGenerationJobPayload always rejected it, leaving the row stuck in queued until startup reconciliation. Persist the BullMQ payload on the outbox row at create-time inside IngestEventsService and EndSessionService, then re-enqueue that canonical payload on retry. Major fixes - prompt-builder: escape server_session_id when interpolating into the XML prompt; previously a session id containing `<`, `&`, or quotes could inject XML into the provider input. - ServerJobQueue: route both worker.on('stalled') and the QueueEvents 'stalled' subscriber through a single notifyStalled helper that dedupes by jobId for 30s, so counters.stalled increments once per stall. QueueEvents 'error' now routes through notifyQueueError so it increments counters.errored and runs onError listeners — keeping observability symmetric across both sources. - ServerV1PostgresRoutes: convert PostgresObservationRepository from three dynamic imports to a single static import for consistency. - mcp-server / ServerBetaClient: actually forward the observation_record_event tool's `generate` flag through to the /v1/events endpoint as `?generate=false` instead of voiding it. - server-sessions.markGenerationFailed: guard jsonb_set against a null error payload so the failure path can't null out metadata before the generation_status='failed' write commits. Minor fixes - server-sessions.endSession: keep updated_at stable on repeated calls so the documented idempotency contract holds. - SettingsDefaultsManager + ServerBetaService.getServerBetaPort: derive the server-beta default port from UID (37877 + uid%100), matching the worker port pattern, so two users on the same host don't collide. Docker stacks always pass CLAUDE_MEM_SERVER_PORT explicitly so the containerized deployment is unaffected. - server-session-runtime test: close the pg.Pool in afterAll. - server-beta-release-readiness.md: escape pipes inside table inline code, add `text` language tag to the fenced log block. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address Greptile + CodeRabbit third review on PR #2383 P1 fixes - SessionsObservationsAdapter.resolveServerSession: catch unique-violation (23505) on concurrent compat inserts and re-fetch instead of returning 500. Two compat callers carrying the same contentSessionId can both observe `existing===null` and race on the (project_id, external_session_id) unique constraint; the second now resolves to the raced row instead of dropping the event. - /v1/events/batch: pass `sourceAdapter: null` to ingestBatch so each event's BullMQ payload (and persisted outbox payload column) reflects its own event.sourceAdapter via buildEventBullmqPayload's fallback, rather than stamping the whole batch with the first event's adapter. Minor - server-session-runtime test afterEach: wrap DROP SCHEMA in try/finally so client.release() always runs even if the drop throws. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): drop `pool as never` cast — pg.Pool already matches PostgresPool Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): retry of completed job now 409s instead of duplicating retryGenerationJob previously fell through to the reset+re-enqueue path when called on a job in `completed` status. The observations index dedupes on (generation_job_id, parsed_observation_index, content) but LLM output is non-deterministic, so a second provider run almost always produced a different content string and bypassed the index, persisting a parallel set of observation rows attributed to the same generation job. Match cancelGenerationJob's 409 guard for completed jobs. failed and cancelled remain valid retry targets. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(server-beta): rebuild bundles after rebase onto main Regenerates the three plugin bundles so they reflect the rebased source state. Mechanical rebuild output only — no source changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): wrap resolveServerSession in try/catch for structured error response Greptile P1 on PR #2383: resolveServerSession was called before the try/catch in both compat adapters, so Postgres errors during session lookup (timeout, pool exhaustion, etc.) escaped to Express's default error handler and returned HTML/text 500s. Legacy clients calling response.json() would get a parse failure instead of the documented { stored: false, reason: 'internal_error' } (or { status: 'error', reason: 'internal_error' } for the summarize adapter) shape. Move the resolveServerSession call inside the existing try block in both adapters so any failure flows through the structured catch handler. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): catch 23505 unique violation in POST /v1/sessions/start Greptile P1 on PR #2383: concurrent requests with the same externalSessionId can both pass the findByExternalIdForScope check, both call repo.create, and the loser hits the (project_id, external_session_id) unique constraint. The handler treated that as an unknown error and returned a 500. Apply the same pattern resolveServerSession already uses: catch error.code '23505' when externalSessionId is set, refetch the row inserted by the winning request, and return 200 with that session. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alex Newman
@@ -0,0 +1,269 @@
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, mock, spyOn } from 'bun:test';
|
||||
import pg from 'pg';
|
||||
import { createHash, randomBytes } from 'crypto';
|
||||
import { Server } from '../../../src/services/server/Server.js';
|
||||
import { ServerV1PostgresRoutes } from '../../../src/server/routes/v1/ServerV1PostgresRoutes.js';
|
||||
import {
|
||||
bootstrapServerBetaPostgresSchema,
|
||||
createPostgresStorageRepositories,
|
||||
type PostgresPoolClient,
|
||||
type PostgresStorageRepositories,
|
||||
} from '../../../src/storage/postgres/index.js';
|
||||
import { DisabledServerBetaQueueManager } from '../../../src/server/runtime/types.js';
|
||||
import { logger } from '../../../src/utils/logger.js';
|
||||
|
||||
// Phase 12 — integration tests for GET /v1/jobs (with admin payload guard),
|
||||
// POST /v1/jobs/:id/retry, POST /v1/jobs/:id/cancel. Postgres-gated; skipped
|
||||
// without CLAUDE_MEM_TEST_POSTGRES_URL.
|
||||
|
||||
const testDatabaseUrl = process.env.CLAUDE_MEM_TEST_POSTGRES_URL;
|
||||
|
||||
function quoteIdentifier(name: string): string {
|
||||
return `"${name.replaceAll('"', '""')}"`;
|
||||
}
|
||||
|
||||
function newApiKey(): { raw: string; hash: string } {
|
||||
const raw = `cm_${randomBytes(24).toString('hex')}`;
|
||||
const hash = createHash('sha256').update(raw).digest('hex');
|
||||
return { raw, hash };
|
||||
}
|
||||
|
||||
describe('Phase 12 — GET /v1/jobs + retry/cancel routes', () => {
|
||||
if (!testDatabaseUrl) {
|
||||
it.skip('requires CLAUDE_MEM_TEST_POSTGRES_URL', () => {});
|
||||
return;
|
||||
}
|
||||
|
||||
let pool: pg.Pool;
|
||||
let client: PostgresPoolClient;
|
||||
let schemaName: string;
|
||||
let storage: PostgresStorageRepositories;
|
||||
let server: Server;
|
||||
let port: number;
|
||||
|
||||
let teamId: string;
|
||||
let projectId: string;
|
||||
let writeKey: string;
|
||||
let adminKey: string;
|
||||
let jobId: string;
|
||||
let loggerSpies: ReturnType<typeof spyOn>[] = [];
|
||||
|
||||
beforeEach(async () => {
|
||||
loggerSpies = [
|
||||
spyOn(logger, 'info').mockImplementation(() => {}),
|
||||
spyOn(logger, 'warn').mockImplementation(() => {}),
|
||||
spyOn(logger, 'error').mockImplementation(() => {}),
|
||||
spyOn(logger, 'debug').mockImplementation(() => {}),
|
||||
];
|
||||
pool = new pg.Pool({ connectionString: testDatabaseUrl });
|
||||
client = await pool.connect();
|
||||
schemaName = `cm_phase12_jobs_${crypto.randomUUID().replaceAll('-', '_')}`;
|
||||
await client.query(`CREATE SCHEMA ${quoteIdentifier(schemaName)}`);
|
||||
await client.query(`SET search_path TO ${quoteIdentifier(schemaName)}`);
|
||||
await bootstrapServerBetaPostgresSchema(client);
|
||||
pool.on('connect', (c) => {
|
||||
c.query(`SET search_path TO ${quoteIdentifier(schemaName)}`).catch(() => {});
|
||||
});
|
||||
storage = createPostgresStorageRepositories(client);
|
||||
|
||||
const team = await storage.teams.create({ name: 'team-a' });
|
||||
const project = await storage.projects.create({ teamId: team.id, name: 'p1' });
|
||||
teamId = team.id;
|
||||
projectId = project.id;
|
||||
|
||||
const writeMaterial = newApiKey();
|
||||
writeKey = writeMaterial.raw;
|
||||
await storage.auth.createApiKey({
|
||||
keyHash: writeMaterial.hash,
|
||||
teamId,
|
||||
projectId: null,
|
||||
actorId: 'system:phase12-write',
|
||||
scopes: ['memories:read', 'memories:write'],
|
||||
});
|
||||
|
||||
const adminMaterial = newApiKey();
|
||||
adminKey = adminMaterial.raw;
|
||||
await storage.auth.createApiKey({
|
||||
keyHash: adminMaterial.hash,
|
||||
teamId,
|
||||
projectId: null,
|
||||
actorId: 'system:phase12-admin',
|
||||
scopes: ['memories:read', 'memories:write', 'memories:admin'],
|
||||
});
|
||||
|
||||
const event = await storage.agentEvents.create({
|
||||
projectId,
|
||||
teamId,
|
||||
sourceAdapter: 'api',
|
||||
eventType: 'tool_use',
|
||||
payload: { sensitive: 'should_not_leak' },
|
||||
occurredAt: new Date(),
|
||||
});
|
||||
const job = await storage.observationGenerationJobs.create({
|
||||
projectId,
|
||||
teamId,
|
||||
sourceType: 'agent_event',
|
||||
sourceId: event.id,
|
||||
agentEventId: event.id,
|
||||
jobType: 'observation_generate_for_event',
|
||||
payload: { sensitive: 'should_not_leak', request_id: 'req-12345' },
|
||||
});
|
||||
jobId = job.id;
|
||||
|
||||
server = new Server({
|
||||
getInitializationComplete: () => true,
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker.cjs',
|
||||
runtime: 'server-beta',
|
||||
getAiStatus: () => ({ provider: 'disabled', authMethod: 'api-key', lastInteraction: null }),
|
||||
});
|
||||
server.registerRoutes(new ServerV1PostgresRoutes({
|
||||
pool: pool as never,
|
||||
queueManager: new DisabledServerBetaQueueManager('disabled in tests'),
|
||||
authMode: 'api-key',
|
||||
runtime: 'server-beta',
|
||||
sessionPolicy: 'per-event',
|
||||
getEventQueue: () => null,
|
||||
getSummaryQueue: () => null,
|
||||
}));
|
||||
server.finalizeRoutes();
|
||||
await server.listen(0, '127.0.0.1');
|
||||
const address = server.getHttpServer()?.address();
|
||||
if (!address || typeof address === 'string') throw new Error('no port');
|
||||
port = address.port;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
try { await server.close(); } catch (error: unknown) {
|
||||
const code = (error as NodeJS.ErrnoException | undefined)?.code;
|
||||
if (code !== 'ERR_SERVER_NOT_RUNNING') throw error;
|
||||
}
|
||||
await client.query(`DROP SCHEMA IF EXISTS ${quoteIdentifier(schemaName)} CASCADE`);
|
||||
client.release();
|
||||
await pool.end();
|
||||
loggerSpies.forEach(spy => spy.mockRestore());
|
||||
mock.restore();
|
||||
});
|
||||
|
||||
function authedFetch(rawKey: string, path: string, init?: RequestInit): Promise<Response> {
|
||||
return fetch(`http://127.0.0.1:${port}${path}`, {
|
||||
...(init ?? {}),
|
||||
headers: {
|
||||
Authorization: `Bearer ${rawKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
...((init?.headers as Record<string, string>) ?? {}),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
it('GET /v1/jobs lists jobs without payload by default', async () => {
|
||||
const resp = await authedFetch(writeKey, '/v1/jobs');
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json() as { jobs: Array<Record<string, unknown>>; total: number };
|
||||
expect(body.total).toBe(1);
|
||||
expect(body.jobs[0]!.payload).toBeUndefined();
|
||||
});
|
||||
|
||||
it('GET /v1/jobs?include=payload rejects without admin scope', async () => {
|
||||
const resp = await authedFetch(writeKey, '/v1/jobs?include=payload');
|
||||
expect(resp.status).toBe(403);
|
||||
});
|
||||
|
||||
it('GET /v1/jobs?include=payload succeeds with admin scope and returns payload', async () => {
|
||||
const resp = await authedFetch(adminKey, '/v1/jobs?include=payload');
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json() as { jobs: Array<Record<string, unknown>> };
|
||||
const payload = body.jobs[0]!.payload as { sensitive: string; request_id?: string };
|
||||
expect(payload.sensitive).toBe('should_not_leak');
|
||||
expect(payload.request_id).toBe('req-12345');
|
||||
});
|
||||
|
||||
it('GET /v1/jobs supports source_type and since filters', async () => {
|
||||
const future = new Date(Date.now() + 60_000).toISOString();
|
||||
const resp = await authedFetch(writeKey, `/v1/jobs?source_type=agent_event&since=${future}`);
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json() as { total: number };
|
||||
expect(body.total).toBe(0);
|
||||
});
|
||||
|
||||
it('POST /v1/jobs/:id/retry on a queued job is a no-op', async () => {
|
||||
const resp = await authedFetch(writeKey, `/v1/jobs/${jobId}/retry`, { method: 'POST' });
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json() as { alreadyQueued: boolean };
|
||||
expect(body.alreadyQueued).toBe(true);
|
||||
// Idempotent: a second call also reports already queued.
|
||||
const resp2 = await authedFetch(writeKey, `/v1/jobs/${jobId}/retry`, { method: 'POST' });
|
||||
expect(resp2.status).toBe(200);
|
||||
const body2 = await resp2.json() as { alreadyQueued: boolean };
|
||||
expect(body2.alreadyQueued).toBe(true);
|
||||
});
|
||||
|
||||
it('POST /v1/jobs/:id/retry on a failed job re-queues idempotently', async () => {
|
||||
// Force the row into failed.
|
||||
await client.query(
|
||||
`UPDATE observation_generation_jobs SET status = 'failed', failed_at = now() WHERE id = $1`,
|
||||
[jobId],
|
||||
);
|
||||
const resp = await authedFetch(writeKey, `/v1/jobs/${jobId}/retry`, { method: 'POST' });
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json() as {
|
||||
alreadyQueued: boolean;
|
||||
retriedCount: number;
|
||||
generationJob: { status: string };
|
||||
};
|
||||
expect(body.alreadyQueued).toBe(false);
|
||||
expect(body.retriedCount).toBe(1);
|
||||
expect(body.generationJob.status).toBe('queued');
|
||||
|
||||
// Second retry on now-queued row is a no-op.
|
||||
const resp2 = await authedFetch(writeKey, `/v1/jobs/${jobId}/retry`, { method: 'POST' });
|
||||
const body2 = await resp2.json() as { alreadyQueued: boolean };
|
||||
expect(body2.alreadyQueued).toBe(true);
|
||||
|
||||
// Audit row written.
|
||||
const audit = await client.query(
|
||||
`SELECT * FROM audit_log WHERE action = 'generation_job.retried_by_operator' AND resource_id = $1`,
|
||||
[jobId],
|
||||
);
|
||||
expect(audit.rows.length).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
it('POST /v1/jobs/:id/cancel cancels a queued job and emits audit', async () => {
|
||||
const resp = await authedFetch(writeKey, `/v1/jobs/${jobId}/cancel`, { method: 'POST' });
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json() as { generationJob: { status: string }; alreadyCancelled: boolean };
|
||||
expect(body.alreadyCancelled).toBe(false);
|
||||
expect(body.generationJob.status).toBe('cancelled');
|
||||
|
||||
// Idempotent.
|
||||
const resp2 = await authedFetch(writeKey, `/v1/jobs/${jobId}/cancel`, { method: 'POST' });
|
||||
const body2 = await resp2.json() as { alreadyCancelled: boolean };
|
||||
expect(body2.alreadyCancelled).toBe(true);
|
||||
|
||||
const audit = await client.query(
|
||||
`SELECT * FROM audit_log WHERE action = 'generation_job.cancelled_by_operator' AND resource_id = $1`,
|
||||
[jobId],
|
||||
);
|
||||
expect(audit.rows.length).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
it('request_id flows from header into audit details', async () => {
|
||||
const resp = await authedFetch(writeKey, '/v1/jobs', {
|
||||
headers: { 'X-Request-Id': 'op-correlation-007' },
|
||||
});
|
||||
expect(resp.status).toBe(200);
|
||||
expect(resp.headers.get('x-request-id')).toBe('op-correlation-007');
|
||||
const body = await resp.json() as { requestId: string };
|
||||
expect(body.requestId).toBe('op-correlation-007');
|
||||
|
||||
const audit = await client.query(
|
||||
`SELECT details FROM audit_log WHERE action = 'observation.read' ORDER BY created_at DESC LIMIT 1`,
|
||||
);
|
||||
const details = audit.rows[0]?.details as { requestId?: string };
|
||||
expect(details?.requestId).toBe('op-correlation-007');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,259 @@
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
//
|
||||
// Phase 8 — verifies the new /v1/memories, /v1/search, /v1/context, and
|
||||
// /v1/jobs/:id REST endpoints behave the way the MCP `observation_*` tools
|
||||
// expect, and verifies the ServerBetaClient (which the MCP tools use) hits
|
||||
// those endpoints end-to-end.
|
||||
//
|
||||
// Postgres-gated: requires CLAUDE_MEM_TEST_POSTGRES_URL.
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, mock, spyOn } from 'bun:test';
|
||||
import pg from 'pg';
|
||||
import { createHash, randomBytes } from 'crypto';
|
||||
import { Server } from '../../../src/services/server/Server.js';
|
||||
import { ServerV1PostgresRoutes } from '../../../src/server/routes/v1/ServerV1PostgresRoutes.js';
|
||||
import {
|
||||
bootstrapServerBetaPostgresSchema,
|
||||
createPostgresStorageRepositories,
|
||||
type PostgresPoolClient,
|
||||
type PostgresStorageRepositories,
|
||||
} from '../../../src/storage/postgres/index.js';
|
||||
import { DisabledServerBetaQueueManager } from '../../../src/server/runtime/types.js';
|
||||
import { ServerBetaClient } from '../../../src/services/hooks/server-beta-client.js';
|
||||
import { logger } from '../../../src/utils/logger.js';
|
||||
|
||||
const testDatabaseUrl = process.env.CLAUDE_MEM_TEST_POSTGRES_URL;
|
||||
|
||||
function quoteIdentifier(name: string): string {
|
||||
return `"${name.replaceAll('"', '""')}"`;
|
||||
}
|
||||
|
||||
function newApiKey(): { raw: string; hash: string } {
|
||||
const raw = `cm_${randomBytes(24).toString('hex')}`;
|
||||
const hash = createHash('sha256').update(raw).digest('hex');
|
||||
return { raw, hash };
|
||||
}
|
||||
|
||||
describe('Phase 8 MCP-backing REST endpoints (/v1/memories, /v1/search, /v1/context, /v1/jobs/:id)', () => {
|
||||
if (!testDatabaseUrl) {
|
||||
it.skip('requires CLAUDE_MEM_TEST_POSTGRES_URL', () => {});
|
||||
return;
|
||||
}
|
||||
|
||||
let pool: pg.Pool;
|
||||
let client: PostgresPoolClient;
|
||||
let schemaName: string;
|
||||
let storage: PostgresStorageRepositories;
|
||||
let server: Server;
|
||||
let port: number;
|
||||
let teamId: string;
|
||||
let projectId: string;
|
||||
let apiKeyRaw: string;
|
||||
let loggerSpies: ReturnType<typeof spyOn>[] = [];
|
||||
|
||||
beforeEach(async () => {
|
||||
loggerSpies = [
|
||||
spyOn(logger, 'info').mockImplementation(() => {}),
|
||||
spyOn(logger, 'warn').mockImplementation(() => {}),
|
||||
spyOn(logger, 'error').mockImplementation(() => {}),
|
||||
spyOn(logger, 'debug').mockImplementation(() => {}),
|
||||
];
|
||||
pool = new pg.Pool({ connectionString: testDatabaseUrl });
|
||||
client = await pool.connect();
|
||||
schemaName = `cm_phase8_routes_${crypto.randomUUID().replaceAll('-', '_')}`;
|
||||
await client.query(`CREATE SCHEMA ${quoteIdentifier(schemaName)}`);
|
||||
await client.query(`SET search_path TO ${quoteIdentifier(schemaName)}`);
|
||||
await bootstrapServerBetaPostgresSchema(client);
|
||||
pool.on('connect', (poolClient) => {
|
||||
poolClient.query(`SET search_path TO ${quoteIdentifier(schemaName)}`).catch(() => {});
|
||||
});
|
||||
storage = createPostgresStorageRepositories(client);
|
||||
|
||||
const team = await storage.teams.create({ name: 'team' });
|
||||
const project = await storage.projects.create({ teamId: team.id, name: 'p' });
|
||||
teamId = team.id;
|
||||
projectId = project.id;
|
||||
|
||||
const { raw, hash } = newApiKey();
|
||||
apiKeyRaw = raw;
|
||||
await storage.auth.createApiKey({
|
||||
keyHash: hash,
|
||||
teamId,
|
||||
projectId,
|
||||
actorId: 'test',
|
||||
scopes: ['memories:read', 'memories:write'],
|
||||
});
|
||||
|
||||
server = new Server({
|
||||
getInitializationComplete: () => true,
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker.cjs',
|
||||
runtime: 'server-beta',
|
||||
getAiStatus: () => ({ provider: 'disabled', authMethod: 'api-key', lastInteraction: null }),
|
||||
});
|
||||
server.registerRoutes(new ServerV1PostgresRoutes({
|
||||
pool: pool as never,
|
||||
queueManager: new DisabledServerBetaQueueManager('disabled in tests'),
|
||||
authMode: 'api-key',
|
||||
runtime: 'server-beta',
|
||||
sessionPolicy: 'per-event',
|
||||
// Capture-only queue stub so /v1/events succeeds without BullMQ.
|
||||
getEventQueue: () => ({
|
||||
async add() {},
|
||||
async getJob() { return null; },
|
||||
async remove() {},
|
||||
}) as never,
|
||||
getSummaryQueue: () => ({
|
||||
async add() {},
|
||||
async getJob() { return null; },
|
||||
async remove() {},
|
||||
}) as never,
|
||||
}));
|
||||
server.finalizeRoutes();
|
||||
await server.listen(0, '127.0.0.1');
|
||||
const address = server.getHttpServer()?.address();
|
||||
if (!address || typeof address === 'string') throw new Error('no port');
|
||||
port = address.port;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
try { await server.close(); } catch (error: unknown) {
|
||||
const code = (error as NodeJS.ErrnoException | undefined)?.code;
|
||||
if (code !== 'ERR_SERVER_NOT_RUNNING') throw error;
|
||||
}
|
||||
await client.query(`DROP SCHEMA IF EXISTS ${quoteIdentifier(schemaName)} CASCADE`);
|
||||
client.release();
|
||||
await pool.end();
|
||||
loggerSpies.forEach(spy => spy.mockRestore());
|
||||
mock.restore();
|
||||
});
|
||||
|
||||
function buildClient(): ServerBetaClient {
|
||||
return new ServerBetaClient({
|
||||
serverBaseUrl: `http://127.0.0.1:${port}`,
|
||||
apiKey: apiKeyRaw,
|
||||
});
|
||||
}
|
||||
|
||||
it('observation_add path: POST /v1/memories inserts an observation without enqueuing generation', async () => {
|
||||
const c = buildClient();
|
||||
const before = await pool.query(`SELECT count(*)::int AS n FROM observation_generation_jobs`);
|
||||
const result = await c.addObservation({
|
||||
projectId,
|
||||
content: 'Manual observation about login bug',
|
||||
kind: 'manual',
|
||||
metadata: { tag: 'mcp' },
|
||||
});
|
||||
expect(result.memory.id).toBeTruthy();
|
||||
expect(result.memory.projectId).toBe(projectId);
|
||||
expect(result.memory.content).toBe('Manual observation about login bug');
|
||||
|
||||
const obsCount = await pool.query(`SELECT count(*)::int AS n FROM observations`);
|
||||
expect(obsCount.rows[0]?.n).toBe(1);
|
||||
|
||||
// Anti-pattern guard: /v1/memories MUST NOT create a generation job.
|
||||
const after = await pool.query(`SELECT count(*)::int AS n FROM observation_generation_jobs`);
|
||||
expect(after.rows[0]?.n).toBe(before.rows[0]?.n);
|
||||
});
|
||||
|
||||
it('observation_record_event path: POST /v1/events creates event row + outbox row atomically', async () => {
|
||||
const c = buildClient();
|
||||
const result = await c.recordEvent({
|
||||
projectId,
|
||||
sourceType: 'api',
|
||||
eventType: 'mcp_test_event',
|
||||
occurredAtEpoch: Date.now(),
|
||||
payload: { hello: 'world' },
|
||||
});
|
||||
expect(result.event.id).toBeTruthy();
|
||||
|
||||
const eventRows = await pool.query(`SELECT id, project_id FROM agent_events`);
|
||||
expect(eventRows.rows).toHaveLength(1);
|
||||
|
||||
// The outbox row should exist because ?generate defaults to true.
|
||||
const jobRows = await pool.query(
|
||||
`SELECT id, source_type, status FROM observation_generation_jobs WHERE source_type = 'agent_event'`,
|
||||
);
|
||||
expect(jobRows.rows).toHaveLength(1);
|
||||
expect(jobRows.rows[0]?.status).toBe('queued');
|
||||
});
|
||||
|
||||
it('observation_search path: POST /v1/search returns FTS-ranked observations from PostgresObservationRepository', async () => {
|
||||
// Seed two observations directly via REST so we exercise the same write path.
|
||||
const c = buildClient();
|
||||
await c.addObservation({ projectId, content: 'Refactored authentication middleware to use JWT verification', kind: 'manual' });
|
||||
await c.addObservation({ projectId, content: 'Fixed flaky test in payment processing', kind: 'manual' });
|
||||
|
||||
const matches = await c.searchObservations({ projectId, query: 'authentication', limit: 10 });
|
||||
expect(matches.observations.length).toBeGreaterThanOrEqual(1);
|
||||
expect(matches.observations[0]?.content).toContain('authentication');
|
||||
|
||||
const noMatches = await c.searchObservations({ projectId, query: 'nonexistent_xyz_term', limit: 10 });
|
||||
expect(noMatches.observations).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('observation_context path: POST /v1/context returns observations + concatenated context', async () => {
|
||||
const c = buildClient();
|
||||
await c.addObservation({ projectId, content: 'first observation about deployment pipeline', kind: 'manual' });
|
||||
await c.addObservation({ projectId, content: 'second observation about deployment pipeline', kind: 'manual' });
|
||||
|
||||
const result = await c.contextObservations({ projectId, query: 'deployment', limit: 5 });
|
||||
expect(result.observations.length).toBeGreaterThanOrEqual(2);
|
||||
expect(result.context).toContain('deployment pipeline');
|
||||
// Context joins observations with a blank line.
|
||||
expect(result.context.split('\n\n').length).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
it('observation_generation_status path: GET /v1/jobs/:id returns the same payload as REST', async () => {
|
||||
const c = buildClient();
|
||||
const recorded = await c.recordEvent({
|
||||
projectId,
|
||||
sourceType: 'api',
|
||||
eventType: 'mcp_status_test',
|
||||
occurredAtEpoch: Date.now(),
|
||||
});
|
||||
const jobId = (recorded.generationJob as { id: string } | undefined)?.id;
|
||||
expect(jobId).toBeTruthy();
|
||||
|
||||
const status = await c.getJobStatus(jobId!);
|
||||
expect(status.generationJob.id).toBe(jobId);
|
||||
expect(status.generationJob.status).toBe('queued');
|
||||
|
||||
// Compare with the raw HTTP response — same payload contract.
|
||||
const raw = await fetch(`http://127.0.0.1:${port}/v1/jobs/${encodeURIComponent(jobId!)}`, {
|
||||
headers: { Authorization: `Bearer ${apiKeyRaw}` },
|
||||
});
|
||||
expect(raw.status).toBe(200);
|
||||
const rawJson = await raw.json();
|
||||
expect(rawJson.generationJob.id).toBe(jobId);
|
||||
});
|
||||
|
||||
it('end-to-end: observation_add → observation_search returns the inserted observation (no provider needed)', async () => {
|
||||
const c = buildClient();
|
||||
const inserted = await c.addObservation({
|
||||
projectId,
|
||||
content: 'End-to-end harness verifies idempotent search round-trip',
|
||||
kind: 'manual',
|
||||
});
|
||||
const found = await c.searchObservations({ projectId, query: 'harness verifies idempotent', limit: 5 });
|
||||
expect(found.observations.some(observation => observation.id === inserted.memory.id)).toBe(true);
|
||||
});
|
||||
|
||||
it('cross-tenant request to /v1/search is rejected', async () => {
|
||||
// Create a foreign project under a different team.
|
||||
const otherTeam = await storage.teams.create({ name: 'foreign' });
|
||||
const otherProject = await storage.projects.create({ teamId: otherTeam.id, name: 'foreign-p' });
|
||||
|
||||
const c = buildClient();
|
||||
let caught: unknown;
|
||||
try {
|
||||
await c.searchObservations({ projectId: otherProject.id, query: 'anything' });
|
||||
} catch (error) {
|
||||
caught = error;
|
||||
}
|
||||
// The api-key is scoped to `projectId`; foreign access yields 403.
|
||||
expect(String(caught)).toContain('403');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,226 @@
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, mock, spyOn } from 'bun:test';
|
||||
import pg from 'pg';
|
||||
import { createHash, randomBytes } from 'crypto';
|
||||
import { Server } from '../../../src/services/server/Server.js';
|
||||
import { ServerV1PostgresRoutes } from '../../../src/server/routes/v1/ServerV1PostgresRoutes.js';
|
||||
import {
|
||||
bootstrapServerBetaPostgresSchema,
|
||||
createPostgresStorageRepositories,
|
||||
type PostgresPoolClient,
|
||||
type PostgresStorageRepositories,
|
||||
} from '../../../src/storage/postgres/index.js';
|
||||
import { DisabledServerBetaQueueManager } from '../../../src/server/runtime/types.js';
|
||||
import { logger } from '../../../src/utils/logger.js';
|
||||
|
||||
const testDatabaseUrl = process.env.CLAUDE_MEM_TEST_POSTGRES_URL;
|
||||
|
||||
function quoteIdentifier(name: string): string {
|
||||
return `"${name.replaceAll('"', '""')}"`;
|
||||
}
|
||||
|
||||
function newApiKey(): { raw: string; hash: string } {
|
||||
const raw = `cm_${randomBytes(24).toString('hex')}`;
|
||||
const hash = createHash('sha256').update(raw).digest('hex');
|
||||
return { raw, hash };
|
||||
}
|
||||
|
||||
describe('ServerV1PostgresRoutes Phase 6 session endpoints', () => {
|
||||
if (!testDatabaseUrl) {
|
||||
it.skip('requires CLAUDE_MEM_TEST_POSTGRES_URL', () => {});
|
||||
return;
|
||||
}
|
||||
|
||||
let pool: pg.Pool;
|
||||
let client: PostgresPoolClient;
|
||||
let schemaName: string;
|
||||
let storage: PostgresStorageRepositories;
|
||||
let server: Server;
|
||||
let port: number;
|
||||
let teamId: string;
|
||||
let projectId: string;
|
||||
let apiKeyRaw: string;
|
||||
let enqueuedEventJobs: { id: string; payload: unknown }[] = [];
|
||||
let enqueuedSummaryJobs: { id: string; payload: unknown }[] = [];
|
||||
let loggerSpies: ReturnType<typeof spyOn>[] = [];
|
||||
|
||||
beforeEach(async () => {
|
||||
loggerSpies = [
|
||||
spyOn(logger, 'info').mockImplementation(() => {}),
|
||||
spyOn(logger, 'warn').mockImplementation(() => {}),
|
||||
spyOn(logger, 'error').mockImplementation(() => {}),
|
||||
spyOn(logger, 'debug').mockImplementation(() => {}),
|
||||
];
|
||||
pool = new pg.Pool({ connectionString: testDatabaseUrl });
|
||||
client = await pool.connect();
|
||||
schemaName = `cm_phase6_routes_${crypto.randomUUID().replaceAll('-', '_')}`;
|
||||
await client.query(`CREATE SCHEMA ${quoteIdentifier(schemaName)}`);
|
||||
await client.query(`SET search_path TO ${quoteIdentifier(schemaName)}`);
|
||||
await bootstrapServerBetaPostgresSchema(client);
|
||||
pool.on('connect', (poolClient) => {
|
||||
poolClient.query(`SET search_path TO ${quoteIdentifier(schemaName)}`).catch(() => {});
|
||||
});
|
||||
storage = createPostgresStorageRepositories(client);
|
||||
|
||||
const team = await storage.teams.create({ name: 'team' });
|
||||
const project = await storage.projects.create({ teamId: team.id, name: 'p' });
|
||||
teamId = team.id;
|
||||
projectId = project.id;
|
||||
|
||||
const { raw, hash } = newApiKey();
|
||||
apiKeyRaw = raw;
|
||||
await storage.auth.createApiKey({
|
||||
keyHash: hash,
|
||||
teamId,
|
||||
projectId,
|
||||
actorId: 'test',
|
||||
scopes: ['memories:read', 'memories:write'],
|
||||
});
|
||||
|
||||
enqueuedEventJobs = [];
|
||||
enqueuedSummaryJobs = [];
|
||||
|
||||
server = new Server({
|
||||
getInitializationComplete: () => true,
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker.cjs',
|
||||
runtime: 'server-beta',
|
||||
getAiStatus: () => ({ provider: 'disabled', authMethod: 'api-key', lastInteraction: null }),
|
||||
});
|
||||
server.registerRoutes(new ServerV1PostgresRoutes({
|
||||
pool: pool as never,
|
||||
queueManager: new DisabledServerBetaQueueManager('disabled in tests'),
|
||||
authMode: 'api-key',
|
||||
runtime: 'server-beta',
|
||||
sessionPolicy: 'per-event',
|
||||
getEventQueue: () => ({
|
||||
async add(jobId: string, payload: unknown) {
|
||||
enqueuedEventJobs.push({ id: jobId, payload });
|
||||
},
|
||||
async getJob() { return null; },
|
||||
async remove() {},
|
||||
}) as never,
|
||||
getSummaryQueue: () => ({
|
||||
async add(jobId: string, payload: unknown) {
|
||||
enqueuedSummaryJobs.push({ id: jobId, payload });
|
||||
},
|
||||
async getJob() { return null; },
|
||||
async remove() {},
|
||||
}) as never,
|
||||
}));
|
||||
server.finalizeRoutes();
|
||||
await server.listen(0, '127.0.0.1');
|
||||
const address = server.getHttpServer()?.address();
|
||||
if (!address || typeof address === 'string') throw new Error('no port');
|
||||
port = address.port;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
try { await server.close(); } catch (error: unknown) {
|
||||
const code = (error as NodeJS.ErrnoException | undefined)?.code;
|
||||
if (code !== 'ERR_SERVER_NOT_RUNNING') throw error;
|
||||
}
|
||||
await client.query(`DROP SCHEMA IF EXISTS ${quoteIdentifier(schemaName)} CASCADE`);
|
||||
client.release();
|
||||
await pool.end();
|
||||
loggerSpies.forEach(spy => spy.mockRestore());
|
||||
mock.restore();
|
||||
});
|
||||
|
||||
function authedFetch(path: string, init: RequestInit = {}): Promise<Response> {
|
||||
return fetch(`http://127.0.0.1:${port}${path}`, {
|
||||
...init,
|
||||
headers: {
|
||||
...(init.headers ?? {}),
|
||||
Authorization: `Bearer ${apiKeyRaw}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
it('POST /v1/sessions/start is idempotent on (project_id, external_session_id)', async () => {
|
||||
const a = await authedFetch('/v1/sessions/start', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ projectId, externalSessionId: 'ext-1' }),
|
||||
});
|
||||
expect(a.status).toBe(201);
|
||||
const aJson = await a.json();
|
||||
const b = await authedFetch('/v1/sessions/start', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ projectId, externalSessionId: 'ext-1' }),
|
||||
});
|
||||
expect(b.status).toBe(200);
|
||||
const bJson = await b.json();
|
||||
expect(bJson.session.id).toBe(aJson.session.id);
|
||||
});
|
||||
|
||||
it('POST /v1/sessions/:id/end enqueues exactly one summary job, idempotent on re-end', async () => {
|
||||
const startResp = await authedFetch('/v1/sessions/start', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ projectId, externalSessionId: 'ext-end' }),
|
||||
});
|
||||
const { session } = await startResp.json();
|
||||
|
||||
const end1 = await authedFetch(`/v1/sessions/${session.id}/end`, { method: 'POST' });
|
||||
expect(end1.status).toBe(200);
|
||||
const end1Json = await end1.json();
|
||||
expect(end1Json.generationJob.sourceType).toBe('session_summary');
|
||||
expect(end1Json.session.endedAtEpoch).not.toBeNull();
|
||||
expect(enqueuedSummaryJobs.length).toBe(1);
|
||||
|
||||
const end2 = await authedFetch(`/v1/sessions/${session.id}/end`, { method: 'POST' });
|
||||
expect(end2.status).toBe(200);
|
||||
const end2Json = await end2.json();
|
||||
// Same generation job id (UNIQUE collapse).
|
||||
expect(end2Json.generationJob.id).toBe(end1Json.generationJob.id);
|
||||
// Re-ending may still publish to the queue (BullMQ add() is idempotent on
|
||||
// jobId), but the outbox row count is unchanged. We assert the outbox
|
||||
// collapse rather than queue-publish count.
|
||||
const allJobs = await storage.observationGenerationJobs.listByStatusForScope({
|
||||
status: 'queued',
|
||||
projectId,
|
||||
teamId,
|
||||
});
|
||||
const summaryJobs = allJobs.filter(j => j.sourceType === 'session_summary');
|
||||
expect(summaryJobs.length).toBe(1);
|
||||
});
|
||||
|
||||
it('GET /v1/sessions/:id returns 404 for cross-project requests', async () => {
|
||||
// Create a foreign project + session under a different team.
|
||||
const otherTeam = await storage.teams.create({ name: 'other' });
|
||||
const otherProject = await storage.projects.create({ teamId: otherTeam.id, name: 'other-p' });
|
||||
const otherSession = await storage.sessions.create({
|
||||
teamId: otherTeam.id,
|
||||
projectId: otherProject.id,
|
||||
externalSessionId: 'foreign',
|
||||
});
|
||||
|
||||
const resp = await authedFetch(`/v1/sessions/${otherSession.id}`);
|
||||
expect(resp.status).toBe(404);
|
||||
});
|
||||
|
||||
it('POST /v1/events with per-event policy enqueues immediately', async () => {
|
||||
const startResp = await authedFetch('/v1/sessions/start', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ projectId, externalSessionId: 'ext-evt' }),
|
||||
});
|
||||
const { session } = await startResp.json();
|
||||
|
||||
const eventResp = await authedFetch('/v1/events', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({
|
||||
projectId,
|
||||
serverSessionId: session.id,
|
||||
sourceType: 'api',
|
||||
eventType: 'tool_use',
|
||||
payload: { tool: 'read' },
|
||||
occurredAtEpoch: Date.now(),
|
||||
}),
|
||||
});
|
||||
expect(eventResp.status).toBe(201);
|
||||
expect(enqueuedEventJobs.length).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,354 @@
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
import { afterAll, afterEach, beforeEach, describe, expect, it } from 'bun:test';
|
||||
import pg from 'pg';
|
||||
import {
|
||||
bootstrapServerBetaPostgresSchema,
|
||||
createPostgresStorageRepositories,
|
||||
PostgresServerSessionsRepository,
|
||||
type PostgresPoolClient,
|
||||
type PostgresStorageRepositories,
|
||||
} from '../../../src/storage/postgres/index.js';
|
||||
import { ServerSessionRuntimeRepository } from '../../../src/server/runtime/ServerSessionRuntimeRepository.js';
|
||||
import {
|
||||
buildEnqueueEventDecision,
|
||||
buildSummaryJobId,
|
||||
resolveSessionGenerationPolicy,
|
||||
} from '../../../src/server/runtime/SessionGenerationPolicy.js';
|
||||
import { processSessionSummaryResponse } from '../../../src/server/generation/processGeneratedResponse.js';
|
||||
|
||||
const testDatabaseUrl = process.env.CLAUDE_MEM_TEST_POSTGRES_URL;
|
||||
|
||||
function quoteIdentifier(name: string): string {
|
||||
return `"${name.replaceAll('"', '""')}"`;
|
||||
}
|
||||
|
||||
describe('SessionGenerationPolicy (pure)', () => {
|
||||
it('defaults to per-event when env is unset', () => {
|
||||
const oldEnv = process.env.CLAUDE_MEM_SERVER_SESSION_POLICY;
|
||||
delete process.env.CLAUDE_MEM_SERVER_SESSION_POLICY;
|
||||
try {
|
||||
const resolved = resolveSessionGenerationPolicy();
|
||||
expect(resolved.policy).toBe('per-event');
|
||||
} finally {
|
||||
if (oldEnv !== undefined) process.env.CLAUDE_MEM_SERVER_SESSION_POLICY = oldEnv;
|
||||
}
|
||||
});
|
||||
|
||||
it('honors explicit policy override', () => {
|
||||
expect(resolveSessionGenerationPolicy({ policy: 'debounce' }).policy).toBe('debounce');
|
||||
expect(resolveSessionGenerationPolicy({ policy: 'end-of-session' }).policy).toBe('end-of-session');
|
||||
expect(resolveSessionGenerationPolicy({ policy: 'per-event' }).policy).toBe('per-event');
|
||||
});
|
||||
|
||||
it('per-event policy enqueues immediately with no delay', () => {
|
||||
const decision = buildEnqueueEventDecision({
|
||||
event: makeFakeEvent('e1', 's1'),
|
||||
outbox: makeFakeOutbox('j1', 'e1'),
|
||||
}, { policy: 'per-event' });
|
||||
expect(decision.shouldEnqueue).toBe(true);
|
||||
expect(decision.reason).toBe('per-event');
|
||||
expect(decision.jobsOptions).toBeUndefined();
|
||||
});
|
||||
|
||||
it('debounce policy enqueues with delay', () => {
|
||||
const decision = buildEnqueueEventDecision({
|
||||
event: makeFakeEvent('e1', 's1'),
|
||||
outbox: makeFakeOutbox('j1', 'e1'),
|
||||
}, { policy: 'debounce', debounceWindowMs: 1234 });
|
||||
expect(decision.shouldEnqueue).toBe(true);
|
||||
expect(decision.reason).toBe('debounce');
|
||||
expect(decision.jobsOptions?.delay).toBe(1234);
|
||||
});
|
||||
|
||||
it('end-of-session policy skips enqueue', () => {
|
||||
const decision = buildEnqueueEventDecision({
|
||||
event: makeFakeEvent('e1', 's1'),
|
||||
outbox: makeFakeOutbox('j1', 'e1'),
|
||||
}, { policy: 'end-of-session' });
|
||||
expect(decision.shouldEnqueue).toBe(false);
|
||||
expect(decision.reason).toBe('end-of-session-skip');
|
||||
});
|
||||
|
||||
it('summary job id is deterministic per server_session_id', () => {
|
||||
const a = buildSummaryJobId({ serverSessionId: 's1', teamId: 't', projectId: 'p' });
|
||||
const b = buildSummaryJobId({ serverSessionId: 's1', teamId: 't', projectId: 'p' });
|
||||
const c = buildSummaryJobId({ serverSessionId: 's2', teamId: 't', projectId: 'p' });
|
||||
expect(a).toBe(b);
|
||||
expect(a).not.toBe(c);
|
||||
expect(a).not.toContain(':');
|
||||
});
|
||||
});
|
||||
|
||||
describe('ServerSessionRuntimeRepository + Postgres', () => {
|
||||
if (!testDatabaseUrl) {
|
||||
it.skip('requires CLAUDE_MEM_TEST_POSTGRES_URL', () => {});
|
||||
return;
|
||||
}
|
||||
|
||||
const pool = new pg.Pool({ connectionString: testDatabaseUrl });
|
||||
let client: PostgresPoolClient;
|
||||
let schemaName: string;
|
||||
let storage: PostgresStorageRepositories;
|
||||
let runtime: ServerSessionRuntimeRepository;
|
||||
let teamId: string;
|
||||
let projectId: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
client = await pool.connect();
|
||||
schemaName = `cm_phase6_${crypto.randomUUID().replaceAll('-', '_')}`;
|
||||
await client.query(`CREATE SCHEMA ${quoteIdentifier(schemaName)}`);
|
||||
await client.query(`SET search_path TO ${quoteIdentifier(schemaName)}`);
|
||||
await bootstrapServerBetaPostgresSchema(client);
|
||||
storage = createPostgresStorageRepositories(client);
|
||||
runtime = new ServerSessionRuntimeRepository({ client });
|
||||
|
||||
const team = await storage.teams.create({ name: 'team' });
|
||||
const project = await storage.projects.create({ teamId: team.id, name: 'p' });
|
||||
teamId = team.id;
|
||||
projectId = project.id;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
if (!client) return;
|
||||
try {
|
||||
if (schemaName) {
|
||||
await client.query(`DROP SCHEMA IF EXISTS ${quoteIdentifier(schemaName)} CASCADE`);
|
||||
}
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await pool.end();
|
||||
});
|
||||
|
||||
it('getActiveSession is idempotent on (project_id, external_session_id)', async () => {
|
||||
const a = await runtime.getActiveSession({
|
||||
teamId,
|
||||
projectId,
|
||||
externalSessionId: 'ext-1',
|
||||
});
|
||||
const b = await runtime.getActiveSession({
|
||||
teamId,
|
||||
projectId,
|
||||
externalSessionId: 'ext-1',
|
||||
});
|
||||
expect(a.id).toBe(b.id);
|
||||
expect(a.externalSessionId).toBe('ext-1');
|
||||
});
|
||||
|
||||
it('endSession is idempotent and never duplicates summary jobs', async () => {
|
||||
const session = await runtime.getActiveSession({
|
||||
teamId,
|
||||
projectId,
|
||||
externalSessionId: 'ext-1',
|
||||
});
|
||||
|
||||
const ended1 = await runtime.endSession({ id: session.id, projectId, teamId });
|
||||
expect(ended1?.endedAtEpoch).not.toBeNull();
|
||||
const firstEndedAt = ended1!.endedAtEpoch;
|
||||
|
||||
// Re-end: should preserve original ended_at because of COALESCE.
|
||||
const ended2 = await runtime.endSession({ id: session.id, projectId, teamId });
|
||||
expect(ended2?.endedAtEpoch).toBe(firstEndedAt);
|
||||
|
||||
// Now create a summary outbox row twice — UNIQUE on
|
||||
// (team_id, project_id, source_type, source_id, job_type) collapses.
|
||||
const job1 = await storage.observationGenerationJobs.create({
|
||||
projectId,
|
||||
teamId,
|
||||
sourceType: 'session_summary',
|
||||
sourceId: session.id,
|
||||
serverSessionId: session.id,
|
||||
jobType: 'observation_generate_session_summary',
|
||||
});
|
||||
const job2 = await storage.observationGenerationJobs.create({
|
||||
projectId,
|
||||
teamId,
|
||||
sourceType: 'session_summary',
|
||||
sourceId: session.id,
|
||||
serverSessionId: session.id,
|
||||
jobType: 'observation_generate_session_summary',
|
||||
});
|
||||
expect(job2.id).toBe(job1.id);
|
||||
});
|
||||
|
||||
it('listUnprocessedEvents excludes events with completed jobs', async () => {
|
||||
const session = await runtime.getActiveSession({
|
||||
teamId,
|
||||
projectId,
|
||||
externalSessionId: 'ext-1',
|
||||
});
|
||||
|
||||
const eventA = await storage.agentEvents.create({
|
||||
projectId,
|
||||
teamId,
|
||||
serverSessionId: session.id,
|
||||
sourceAdapter: 'api',
|
||||
eventType: 'tool_use',
|
||||
payload: { x: 1 },
|
||||
occurredAt: new Date(Date.now() - 2000),
|
||||
});
|
||||
const eventB = await storage.agentEvents.create({
|
||||
projectId,
|
||||
teamId,
|
||||
serverSessionId: session.id,
|
||||
sourceAdapter: 'api',
|
||||
eventType: 'tool_use',
|
||||
payload: { x: 2 },
|
||||
occurredAt: new Date(),
|
||||
});
|
||||
|
||||
// Create a job for eventA and mark it completed.
|
||||
const completedJob = await storage.observationGenerationJobs.create({
|
||||
projectId,
|
||||
teamId,
|
||||
sourceType: 'agent_event',
|
||||
sourceId: eventA.id,
|
||||
agentEventId: eventA.id,
|
||||
serverSessionId: session.id,
|
||||
jobType: 'observation_generate_for_event',
|
||||
});
|
||||
await storage.observationGenerationJobs.transitionStatus({
|
||||
id: completedJob.id,
|
||||
projectId,
|
||||
teamId,
|
||||
status: 'processing',
|
||||
});
|
||||
await storage.observationGenerationJobs.transitionStatus({
|
||||
id: completedJob.id,
|
||||
projectId,
|
||||
teamId,
|
||||
status: 'completed',
|
||||
});
|
||||
|
||||
const unprocessed = await runtime.listUnprocessedEvents({
|
||||
teamId,
|
||||
projectId,
|
||||
serverSessionId: session.id,
|
||||
});
|
||||
expect(unprocessed.map(e => e.id)).toEqual([eventB.id]);
|
||||
});
|
||||
|
||||
it('cross-tenant getById returns null', async () => {
|
||||
const otherTeam = await storage.teams.create({ name: 'other' });
|
||||
const otherProject = await storage.projects.create({ teamId: otherTeam.id, name: 'other-p' });
|
||||
const otherSession = await new PostgresServerSessionsRepository(client).create({
|
||||
teamId: otherTeam.id,
|
||||
projectId: otherProject.id,
|
||||
externalSessionId: 'other-1',
|
||||
});
|
||||
|
||||
// Trying to read other team's session under our scope returns null.
|
||||
const result = await runtime.getById({
|
||||
id: otherSession.id,
|
||||
teamId,
|
||||
projectId,
|
||||
});
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
|
||||
it('processSessionSummaryResponse persists kind=summary observation idempotently', async () => {
|
||||
const session = await runtime.getActiveSession({
|
||||
teamId,
|
||||
projectId,
|
||||
externalSessionId: 'ext-summary',
|
||||
});
|
||||
const job = await storage.observationGenerationJobs.create({
|
||||
projectId,
|
||||
teamId,
|
||||
sourceType: 'session_summary',
|
||||
sourceId: session.id,
|
||||
serverSessionId: session.id,
|
||||
jobType: 'observation_generate_session_summary',
|
||||
});
|
||||
await storage.observationGenerationJobs.transitionStatus({
|
||||
id: job.id,
|
||||
projectId,
|
||||
teamId,
|
||||
status: 'processing',
|
||||
});
|
||||
|
||||
const summaryXml = `<summary>
|
||||
<request>investigate session</request>
|
||||
<investigated>queries and traces</investigated>
|
||||
<learned>system behavior</learned>
|
||||
<completed>analysis</completed>
|
||||
<next_steps>plan refactor</next_steps>
|
||||
<notes>none</notes>
|
||||
</summary>`;
|
||||
|
||||
const outcome1 = await processSessionSummaryResponse({
|
||||
pool,
|
||||
job,
|
||||
rawText: summaryXml,
|
||||
providerLabel: 'claude',
|
||||
});
|
||||
expect(outcome1.kind).toBe('completed');
|
||||
if (outcome1.kind === 'completed') {
|
||||
expect(outcome1.observations.length).toBeGreaterThan(0);
|
||||
expect(outcome1.observations[0]!.kind).toBe('summary');
|
||||
}
|
||||
|
||||
// Idempotent: replaying does not produce new observations because the
|
||||
// job is already in completed state.
|
||||
const outcome2 = await processSessionSummaryResponse({
|
||||
pool,
|
||||
job,
|
||||
rawText: summaryXml,
|
||||
providerLabel: 'claude',
|
||||
});
|
||||
expect(outcome2.kind).toBe('completed');
|
||||
if (outcome2.kind === 'completed') {
|
||||
expect(outcome2.observations.length).toBe(0);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
function makeFakeEvent(id: string, sessionId: string | null) {
|
||||
return {
|
||||
id,
|
||||
projectId: 'p',
|
||||
teamId: 't',
|
||||
serverSessionId: sessionId,
|
||||
sourceAdapter: 'api',
|
||||
sourceEventId: null,
|
||||
idempotencyKey: 'k',
|
||||
eventType: 'tool_use',
|
||||
payload: {},
|
||||
metadata: {},
|
||||
occurredAtEpoch: 0,
|
||||
receivedAtEpoch: 0,
|
||||
createdAtEpoch: 0,
|
||||
};
|
||||
}
|
||||
|
||||
function makeFakeOutbox(id: string, eventId: string) {
|
||||
return {
|
||||
id,
|
||||
projectId: 'p',
|
||||
teamId: 't',
|
||||
agentEventId: eventId,
|
||||
sourceType: 'agent_event' as const,
|
||||
sourceId: eventId,
|
||||
serverSessionId: null,
|
||||
jobType: 'observation_generate_for_event',
|
||||
status: 'queued' as const,
|
||||
idempotencyKey: 'k',
|
||||
bullmqJobId: null,
|
||||
attempts: 0,
|
||||
maxAttempts: 3,
|
||||
nextAttemptAtEpoch: null,
|
||||
lockedAtEpoch: null,
|
||||
lockedBy: null,
|
||||
completedAtEpoch: null,
|
||||
failedAtEpoch: null,
|
||||
cancelledAtEpoch: null,
|
||||
lastError: null,
|
||||
payload: {},
|
||||
createdAtEpoch: 0,
|
||||
updatedAtEpoch: 0,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,246 @@
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, mock, spyOn } from 'bun:test';
|
||||
import pg from 'pg';
|
||||
import { createHash, randomBytes } from 'crypto';
|
||||
import { Server } from '../../../src/services/server/Server.js';
|
||||
import { ServerV1PostgresRoutes } from '../../../src/server/routes/v1/ServerV1PostgresRoutes.js';
|
||||
import {
|
||||
bootstrapServerBetaPostgresSchema,
|
||||
createPostgresStorageRepositories,
|
||||
type PostgresPoolClient,
|
||||
type PostgresStorageRepositories,
|
||||
} from '../../../src/storage/postgres/index.js';
|
||||
import { DisabledServerBetaQueueManager } from '../../../src/server/runtime/types.js';
|
||||
import { logger } from '../../../src/utils/logger.js';
|
||||
|
||||
const testDatabaseUrl = process.env.CLAUDE_MEM_TEST_POSTGRES_URL;
|
||||
|
||||
function quoteIdentifier(name: string): string {
|
||||
return `"${name.replaceAll('"', '""')}"`;
|
||||
}
|
||||
|
||||
function newApiKey(): { raw: string; hash: string } {
|
||||
const raw = `cm_${randomBytes(24).toString('hex')}`;
|
||||
const hash = createHash('sha256').update(raw).digest('hex');
|
||||
return { raw, hash };
|
||||
}
|
||||
|
||||
describe('Phase 11 — team/project queue listing endpoints', () => {
|
||||
if (!testDatabaseUrl) {
|
||||
it.skip('requires CLAUDE_MEM_TEST_POSTGRES_URL', () => {});
|
||||
return;
|
||||
}
|
||||
|
||||
let pool: pg.Pool;
|
||||
let client: PostgresPoolClient;
|
||||
let schemaName: string;
|
||||
let storage: PostgresStorageRepositories;
|
||||
let server: Server;
|
||||
let port: number;
|
||||
|
||||
// Tenant scaffolding: two teams, two projects in team-A, one project in
|
||||
// team-B. Three api keys: team-A team-scoped, team-A project-1-scoped,
|
||||
// team-B team-scoped.
|
||||
let teamAId: string;
|
||||
let teamBId: string;
|
||||
let projectA1Id: string;
|
||||
let projectA2Id: string;
|
||||
let projectB1Id: string;
|
||||
let teamAKey: string;
|
||||
let projectA1Key: string;
|
||||
let teamBKey: string;
|
||||
let loggerSpies: ReturnType<typeof spyOn>[] = [];
|
||||
|
||||
beforeEach(async () => {
|
||||
loggerSpies = [
|
||||
spyOn(logger, 'info').mockImplementation(() => {}),
|
||||
spyOn(logger, 'warn').mockImplementation(() => {}),
|
||||
spyOn(logger, 'error').mockImplementation(() => {}),
|
||||
spyOn(logger, 'debug').mockImplementation(() => {}),
|
||||
];
|
||||
pool = new pg.Pool({ connectionString: testDatabaseUrl });
|
||||
client = await pool.connect();
|
||||
schemaName = `cm_phase11_routes_${crypto.randomUUID().replaceAll('-', '_')}`;
|
||||
await client.query(`CREATE SCHEMA ${quoteIdentifier(schemaName)}`);
|
||||
await client.query(`SET search_path TO ${quoteIdentifier(schemaName)}`);
|
||||
await bootstrapServerBetaPostgresSchema(client);
|
||||
pool.on('connect', (poolClient) => {
|
||||
poolClient.query(`SET search_path TO ${quoteIdentifier(schemaName)}`).catch(() => {});
|
||||
});
|
||||
storage = createPostgresStorageRepositories(client);
|
||||
|
||||
const teamA = await storage.teams.create({ name: 'team-a' });
|
||||
const teamB = await storage.teams.create({ name: 'team-b' });
|
||||
const projectA1 = await storage.projects.create({ teamId: teamA.id, name: 'p-a-1' });
|
||||
const projectA2 = await storage.projects.create({ teamId: teamA.id, name: 'p-a-2' });
|
||||
const projectB1 = await storage.projects.create({ teamId: teamB.id, name: 'p-b-1' });
|
||||
teamAId = teamA.id;
|
||||
teamBId = teamB.id;
|
||||
projectA1Id = projectA1.id;
|
||||
projectA2Id = projectA2.id;
|
||||
projectB1Id = projectB1.id;
|
||||
|
||||
const teamAKeyMaterial = newApiKey();
|
||||
teamAKey = teamAKeyMaterial.raw;
|
||||
await storage.auth.createApiKey({
|
||||
keyHash: teamAKeyMaterial.hash,
|
||||
teamId: teamAId,
|
||||
projectId: null,
|
||||
actorId: 'system:phase11-team-a-key',
|
||||
scopes: ['memories:read', 'memories:write'],
|
||||
});
|
||||
|
||||
const projectA1KeyMaterial = newApiKey();
|
||||
projectA1Key = projectA1KeyMaterial.raw;
|
||||
await storage.auth.createApiKey({
|
||||
keyHash: projectA1KeyMaterial.hash,
|
||||
teamId: teamAId,
|
||||
projectId: projectA1Id,
|
||||
actorId: 'system:phase11-project-a1-key',
|
||||
scopes: ['memories:read', 'memories:write'],
|
||||
});
|
||||
|
||||
const teamBKeyMaterial = newApiKey();
|
||||
teamBKey = teamBKeyMaterial.raw;
|
||||
await storage.auth.createApiKey({
|
||||
keyHash: teamBKeyMaterial.hash,
|
||||
teamId: teamBId,
|
||||
projectId: null,
|
||||
actorId: 'system:phase11-team-b-key',
|
||||
scopes: ['memories:read'],
|
||||
});
|
||||
|
||||
// Seed two events in projectA1, one in projectA2, one in projectB1.
|
||||
// Each event creates a generation_jobs row via storage.observationGenerationJobs.
|
||||
for (const projectId of [projectA1Id, projectA1Id, projectA2Id, projectB1Id]) {
|
||||
const teamForProject = projectId === projectB1Id ? teamBId : teamAId;
|
||||
const event = await storage.agentEvents.create({
|
||||
projectId,
|
||||
teamId: teamForProject,
|
||||
sourceAdapter: 'api',
|
||||
eventType: 'tool_use',
|
||||
payload: { p: projectId },
|
||||
occurredAt: new Date(),
|
||||
});
|
||||
await storage.observationGenerationJobs.create({
|
||||
projectId,
|
||||
teamId: teamForProject,
|
||||
sourceType: 'agent_event',
|
||||
sourceId: event.id,
|
||||
agentEventId: event.id,
|
||||
jobType: 'observation_generate_for_event',
|
||||
});
|
||||
}
|
||||
|
||||
server = new Server({
|
||||
getInitializationComplete: () => true,
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker.cjs',
|
||||
runtime: 'server-beta',
|
||||
getAiStatus: () => ({ provider: 'disabled', authMethod: 'api-key', lastInteraction: null }),
|
||||
});
|
||||
server.registerRoutes(new ServerV1PostgresRoutes({
|
||||
pool: pool as never,
|
||||
queueManager: new DisabledServerBetaQueueManager('disabled in tests'),
|
||||
authMode: 'api-key',
|
||||
runtime: 'server-beta',
|
||||
sessionPolicy: 'per-event',
|
||||
getEventQueue: () => null,
|
||||
getSummaryQueue: () => null,
|
||||
}));
|
||||
server.finalizeRoutes();
|
||||
await server.listen(0, '127.0.0.1');
|
||||
const address = server.getHttpServer()?.address();
|
||||
if (!address || typeof address === 'string') throw new Error('no port');
|
||||
port = address.port;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
try { await server.close(); } catch (error: unknown) {
|
||||
const code = (error as NodeJS.ErrnoException | undefined)?.code;
|
||||
if (code !== 'ERR_SERVER_NOT_RUNNING') throw error;
|
||||
}
|
||||
await client.query(`DROP SCHEMA IF EXISTS ${quoteIdentifier(schemaName)} CASCADE`);
|
||||
client.release();
|
||||
await pool.end();
|
||||
loggerSpies.forEach(spy => spy.mockRestore());
|
||||
mock.restore();
|
||||
});
|
||||
|
||||
function authedFetch(rawKey: string, path: string): Promise<Response> {
|
||||
return fetch(`http://127.0.0.1:${port}${path}`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${rawKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
it('GET /v1/teams/:id/jobs returns ALL jobs for the team when called by team-scoped key', async () => {
|
||||
const resp = await authedFetch(teamAKey, `/v1/teams/${teamAId}/jobs`);
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json();
|
||||
// 2 jobs in projectA1 + 1 job in projectA2 = 3
|
||||
expect(body.total).toBe(3);
|
||||
expect(body.jobs.length).toBe(3);
|
||||
expect(body.jobs.every((j: any) => j.teamId === teamAId)).toBe(true);
|
||||
});
|
||||
|
||||
it('GET /v1/teams/:id/jobs returns 404 when caller is from a different team', async () => {
|
||||
const resp = await authedFetch(teamBKey, `/v1/teams/${teamAId}/jobs`);
|
||||
expect(resp.status).toBe(404);
|
||||
});
|
||||
|
||||
it('GET /v1/teams/:id/jobs filters to project scope when caller is project-scoped', async () => {
|
||||
const resp = await authedFetch(projectA1Key, `/v1/teams/${teamAId}/jobs`);
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json();
|
||||
expect(body.total).toBe(2);
|
||||
expect(body.jobs.every((j: any) => j.projectId === projectA1Id)).toBe(true);
|
||||
});
|
||||
|
||||
it('GET /v1/projects/:id/jobs returns 404 when project belongs to another team', async () => {
|
||||
const resp = await authedFetch(teamAKey, `/v1/projects/${projectB1Id}/jobs`);
|
||||
expect(resp.status).toBe(404);
|
||||
});
|
||||
|
||||
it('GET /v1/projects/:id/jobs returns 404 when project-scoped key requests another project', async () => {
|
||||
const resp = await authedFetch(projectA1Key, `/v1/projects/${projectA2Id}/jobs`);
|
||||
expect(resp.status).toBe(404);
|
||||
});
|
||||
|
||||
it('GET /v1/projects/:id/jobs allows project-scoped key to read its own project', async () => {
|
||||
const resp = await authedFetch(projectA1Key, `/v1/projects/${projectA1Id}/jobs`);
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json();
|
||||
expect(body.total).toBe(2);
|
||||
expect(body.jobs.every((j: any) => j.projectId === projectA1Id)).toBe(true);
|
||||
});
|
||||
|
||||
it('GET /v1/projects/:id/jobs allows team-scoped key to read any project under its team', async () => {
|
||||
const resp = await authedFetch(teamAKey, `/v1/projects/${projectA2Id}/jobs`);
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json();
|
||||
expect(body.total).toBe(1);
|
||||
expect(body.jobs.every((j: any) => j.projectId === projectA2Id)).toBe(true);
|
||||
});
|
||||
|
||||
it('supports status filter, limit, and offset', async () => {
|
||||
const resp = await authedFetch(teamAKey, `/v1/teams/${teamAId}/jobs?status=queued&limit=2&offset=0`);
|
||||
expect(resp.status).toBe(200);
|
||||
const body = await resp.json();
|
||||
expect(body.total).toBe(3);
|
||||
expect(body.jobs.length).toBe(2);
|
||||
expect(body.limit).toBe(2);
|
||||
expect(body.offset).toBe(0);
|
||||
expect(body.jobs.every((j: any) => j.status === 'queued')).toBe(true);
|
||||
});
|
||||
|
||||
it('rejects unauthenticated requests', async () => {
|
||||
const resp = await fetch(`http://127.0.0.1:${port}/v1/teams/${teamAId}/jobs`);
|
||||
expect(resp.status).toBe(401);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user