109 Commits

Author	SHA1	Message	Date
Alex Newman	e4ec654355	chore: bump version to 13.3.0	2026-05-21 03:25:35 -07:00
Alex Newman	e7bbb2a9aa	server-beta: Phases 4–13 — event pipeline, generation, MCP, compat, Docker, team audit, observability (#2383) ... * feat(server-beta): Phase 4 — Postgres event-to-generation-job pipeline Adds POST /v1/events, /v1/events/batch, GET /v1/jobs/:id, GET /v1/events/:id, and POST /v1/memories on the server-beta runtime, backed by Postgres. - Event row + outbox generation-job row insert in one withPostgresTransaction. - BullMQ enqueue happens after commit; enqueue failure leaves the row queued for Phase 3 startup reconciliation. - ?generate=false skips the outbox; ?wait=true returns queue status only, never observation IDs (provider generation is Phase 5). - Batch pre-validates all event projectIds against api-key scope before any write; mixed-project batches reject 403 with zero side effects. - /v1/memories is a direct insert alias — no generator, no outbox. - Cross-tenant /v1/jobs/:id returns 404 to avoid leaking row existence. - New PostgresAuthMiddleware reads api_keys by SHA-256 hash; populates req.authContext.teamId/projectId; legacy ServerV1Routes (SQLite, used by worker runtime) is left untouched. - Tests: unit suite hardened with stubbed pool.query so route registration is safe; integration tests skip cleanly without CLAUDE_MEM_TEST_POSTGRES_URL. Verification: 87 pass / 1 skip / 0 fail. No new typecheck errors. Required greps for WorkerService and MemoryItemsRepository in src/server/routes/v1 and src/server/runtime return no hits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 5 — provider observation generator Adds independent provider generation under src/server/generation/ with no worker coupling. Server beta can now generate observations end-to-end: event -> outbox -> BullMQ -> provider -> parser -> persisted observation. - ProviderObservationGenerator orchestrates: lock outbox (queued -> processing), reload agent_event from Postgres (BullMQ payload is advisory only), call provider, hand raw text to processGeneratedResponse, route errors via markGenerationFailed with retryable flag from ServerClassifiedProviderError. - processGeneratedResponse parses with parseAgentXml, persists via PostgresObservationRepository with deterministic generation_key = generation:v1:{job_id}:{index}:{fingerprint}, links via PostgresObservationSourcesRepository, advances outbox status, appends observation_generation_job_events, audits — all in one withPostgresTransaction. Idempotent on retry via UNIQUE constraints. - Three provider adapters under src/server/generation/providers/: Claude, Gemini, OpenRouter. Self-contained — no imports from src/services/worker/*. Worker providers unchanged. - Shared error classification + prompt builder under providers/shared/. Prompt builder strips <private> at the edge; fully-private batches emit <skip_summary /> without billing the provider. - ActiveServerBetaGenerationWorkerManager wires BullMQ Worker via ServerJobQueue.start(...) with concurrency 1 + autorun:false + worker.on('error') per BullMQ docs. - New GET /v1/events/:id/observations on ServerV1PostgresRoutes returns observations linked via observation_sources, team/project scoped. Verification: 104 pass / 4 skip / 0 fail. No typecheck regressions. Anti-pattern greps clean for services/worker imports under src/server, WorkerRef/ActiveSession/SessionStore in src/server/generation. Deferred: ModeManager loading uses a stable fallback observation type list; summary and reindex queue lanes are not yet wired. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 6 — independent server session semantics server_sessions is now the canonical Server beta session model. Sessions are independent of legacy worker ActiveSession state. - PostgresServerSessionRepository extended: findByExternalIdForScope, endSession (idempotent via COALESCE(ended_at, now())), markGenerationStarted/Completed/Failed, listUnprocessedEvents (filters agent_events with completed agent_event jobs). - ServerSessionRuntimeRepository wraps the repo; every method requires explicit team_id + project_id and validates scope via assertProjectOwnership. - SessionGenerationPolicy supports per-event (default), debounce (BullMQ delayed-job replace via getJob+remove+add), and end-of-session. Configured via CLAUDE_MEM_SERVER_SESSION_POLICY and CLAUDE_MEM_SERVER_SESSION_DEBOUNCE_MS env vars; per-team override hooks are exposed on ServerV1PostgresRoutesOptions for future settings layer. - POST /v1/sessions/start (find-or-create on (project_id, external_session_id), GET /v1/sessions/:id (scoped 404), POST /v1/sessions/:id/end (transactional: end + create summary outbox via UNIQUE collapse + enqueue post-commit). Re-ending is fully idempotent. - processSessionSummaryResponse persists summary as kind='summary' observation with the same idempotency model (generation_key + observation_sources UNIQUE). - ProviderObservationGenerator dispatches on source_type: agent_event -> processGeneratedResponse, session_summary -> processSessionSummaryResponse; loadEvents handles session-summary by loading unprocessed events. - ActiveServerBetaGenerationWorkerManager wires summary BullMQ lane alongside event lane (concurrency=1, autorun=false, error listener attached per BullMQ docs). Verification: 110 pass / 6 skip / 0 fail. Net typecheck error count unchanged at 24 (pre-existing, none in Phase 6 files). Anti-pattern greps clean for ActiveSession/SessionStore in src/server/runtime, no worker imports anywhere in src/server. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 7 — hook routing without worker dependency Hooks can now talk directly to server-beta when CLAUDE_MEM_RUNTIME=server-beta is selected, with a clean worker fallback when server-beta is unhealthy. - src/services/hooks/server-beta-client.ts — typed HTTP client for /v1/sessions/start, /v1/events, /v1/sessions/:id/end. Throws ServerBetaClientError with kind classification (missing_api_key, transport, timeout, http_error, invalid_response) and isFallbackEligible helper. Zero imports from services/worker/. - src/services/hooks/runtime-selector.ts — reads CLAUDE_MEM_RUNTIME from settings, returns worker or server-beta context, logs [server-beta-fallback] reason=<code> on every config-time fallback. - src/services/hooks/server-beta-bootstrap.ts — Postgres-backed API key bootstrap. Find-or-creates local-hook-team + local-hook-project, generates cmem_<random> key (SHA-256 hashed), inserts into api_keys with scopes events:write/sessions:write/observations:read/jobs:read. Settings file written with chmod 0600. rotateServerBetaApiKey() wired to a new `claude-mem server keys rotate` command. - src/cli/handlers/{observation,session-init,summarize}.ts — every hook handler tries server-beta first when configured, falls through to the existing worker path on transport/5xx/429/missing-key. One WARN line per fallback. Hook JSON output shape unchanged. - src/shared/SettingsDefaultsManager.ts — three new keys with defaults: CLAUDE_MEM_SERVER_BETA_URL, CLAUDE_MEM_SERVER_BETA_API_KEY, CLAUDE_MEM_SERVER_BETA_PROJECT_ID. - src/npx-cli/commands/install.ts — when installer selects server-beta runtime and CLAUDE_MEM_SERVER_DATABASE_URL is set, bootstraps a local API key automatically. Warns and continues if the DB URL is missing. plugin/scripts/*.cjs bundles rebuilt via npm run build to pick up the new hook handler code path. No plaintext keys in the bundle (verified). Verification: 16 hook unit tests pass; 275 server/storage/services tests pass with 7 pre-existing failures (verified independent of this change via git stash --include-untracked). Build clean. No new typecheck errors in Phase 7 files. Anti-pattern guards verified: - /api/sessions/observations only reached via explicit fallback path - server-beta runtime never starts the worker process - API keys live only in ~/.claude-mem/settings.json (chmod 0600), never in the bundle (grep confirmed) - Worker fallback preserved, observable via single WARN line per call Deferred: semantic context injection (UserPromptSubmit hook) stays worker-only; server-beta does not yet expose /v1/context/semantic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 8 — MCP backed by server-beta core MCP tools now route through server-beta in server-beta mode while keeping worker-mode search/timeline/get_observations tools fully working. - src/servers/mcp-server.ts — five new observation_* tools registered: observation_add, observation_record_event, observation_search, observation_context, observation_generation_status. Three memory_* compatibility aliases delegate to the canonical handlers. Worker auto-start is gated when selectRuntime() === 'server-beta' so MCP in server-beta mode never spawns the worker. - src/services/hooks/server-beta-client.ts — addObservation, searchObservations, contextObservations, getJobStatus added so MCP shares one transport with hooks (Phase 7). - src/server/routes/v1/ServerV1PostgresRoutes.ts — POST /v1/search and POST /v1/context REST cores backed by PostgresObservationRepository full-text search (GIN tsvector from Phase 1). - Existing memory_search/timeline/get_observations tools call callWorkerAPI unchanged in worker mode; worker tests unaffected. Verification: 39 pass / 4 skip / 0 fail on targeted suite. Pre-existing 7 baseline failures verified independent (git stash). No new typecheck errors. WorkerService grep clean across src/servers/mcp-server.ts and src/server/. Anti-pattern guards verified: - No duplicate generation logic in MCP — observation_record_event hits /v1/events which owns event+outbox+enqueue inside one tx - WorkerService not imported anywhere under MCP server-beta path - No hardcoded worker URLs — all transport via Phase 7 ServerBetaClient - memory_* aliases retained, single handler per pair Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 9 — compatibility adapters without coupling Legacy /api/sessions/observations and /api/sessions/summarize endpoints keep working on server-beta runtime by translating to AgentEvent and session-end calls — no worker code, no route duplication. - src/server/services/IngestEventsService.ts — shared event-ingest path used by both /v1/events and the compat adapter. Owns transactional event row + outbox row + lifecycle log + post-commit BullMQ enqueue, honors Phase 6 SessionGenerationPolicy. - src/server/services/EndSessionService.ts — shared session-end path used by both /v1/sessions/:id/end and the compat adapter. Idempotent ended_at + summary outbox + deterministic summary job id. - src/server/compat/SessionsObservationsAdapter.ts — translates legacy POST /api/sessions/observations payload (Claude Code transcript shape) -> AgentEvent (source_adapter='claude-code-compat', event_type='tool_use') -> IngestEventsService.ingestOne. Resolves contentSessionId to server_sessions via find-or-create. - src/server/compat/SessionsSummarizeAdapter.ts — translates legacy POST /api/sessions/summarize -> EndSessionService.end. Preserves the legacy agentId -> {status:'skipped', reason:'subagent_context'} behavior so existing clients see the same response shape. - src/server/routes/v1/ServerV1PostgresRoutes.ts — refactored to delegate to the new shared services (-203 LoC net) so /v1 and /api compat both call the SAME canonical code path. - src/server/runtime/ServerBetaService.ts — registers both compat adapters alongside ServerV1PostgresRoutes, sharing service instances. - docs/server-beta-parity-map.md — full enumeration of legacy /api/* routes labeled native, adapter, or unsupported (with reasons). Viewer read-path adapters explicitly listed as unsupported pending a future viewer-rewrite phase. Verification: 7 compat tests pass, 6 v1-routes tests still pass (refactor preserved behavior), 4 session-routes tests pass. Pre- existing 16 baseline failures verified independent via git stash. Zero new typecheck errors. Anti-pattern guards verified: - No services/worker/http/routes or WorkerService imports under src/server/compat or src/server/runtime - Compat adapters are thin translators with names ending in *Adapter and a top-of-file comment noting they are legacy compatibility - /v1/* remains the canonical Server beta API; compat adapters call shared services rather than acting as a parallel API Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 10 — Docker stack and deployable runtime Server beta now ships as a Docker stack with no worker process anywhere and a separate horizontal generation worker for scaling. - src/server/runtime/create-server-beta-service.ts — validateServerBetaEnv() fails fast on missing CLAUDE_MEM_SERVER_DATABASE_URL, requires CLAUDE_MEM_QUEUE_ENGINE=bullmq in Docker, rejects CLAUDE_MEM_AUTH_MODE=local-dev and CLAUDE_MEM_ALLOW_LOCAL_DEV_BYPASS inside containers (detected via /.dockerenv or CLAUDE_MEM_DOCKER=1). Adds CLAUDE_MEM_GENERATION_DISABLED so the HTTP service can run generator-free. - src/server/runtime/ServerBetaService.ts — runServerBetaGenerationWorker for the dedicated consumer process; runServerBetaApiKeyCli is a new Postgres-backed `server api-key` command (the legacy worker CLI wrote to SQLite and was invisible to the Postgres runtime); getQueueHealth shim feeds /api/health a consistent ObservationQueueHealth shape. - src/npx-cli/commands/{runtime,server}.ts — `claude-mem server worker start` subcommand that boots only the BullMQ consumer. - docker/claude-mem/{Dockerfile,entrypoint.sh} — entrypoint forces CLAUDE_MEM_DOCKER=1 + CLAUDE_MEM_RUNTIME=server-beta and exposes three modes: server (HTTP only, generation disabled), worker (BullMQ consumer), shell. Worker bundle is no longer the default CMD. - docker-compose.yml — full stack: postgres + valkey + claude-mem-server (HTTP-only) + claude-mem-worker (generation consumer). Wires service-to-service env vars. - scripts/e2e-server-beta-docker.sh + docker/e2e/server-beta-e2e.mjs — E2E now hits /v1/sessions/start, /v1/events?wait=true, /v1/jobs/:id; asserts no worker-service.cjs process anywhere in the stack; one-shot docker compose run --rm verifies local-dev auth is rejected with the expected stderr; restart-and-verify confirms Postgres durability and BullMQ retry idempotency. - docs/server.md — full Phase 10 doc: stack diagram, env table, worker mode, auth-in-Docker policy. - docs/api.md — event generation semantics (wait=true, generationJob). Verification: full Docker E2E PASSED on live daemon (phase1 + phase2 + restart-and-verify + revoked-key + no-worker- process + local-dev-rejected). Unit tests 292 pass / 9 skip / 7 fail (7 fails pre-existing baseline). Zero new typecheck errors. Anti-pattern guards verified: - entrypoint never execs worker-service.cjs; E2E greps prove no worker process anywhere in the stack - validateServerBetaEnv refuses local-dev auth in Docker with explicit remediation message; ALLOW_LOCAL_DEV_BYPASS rejected the same way - Docker requires CLAUDE_MEM_QUEUE_ENGINE=bullmq; in-process queue rejected at startup - claude-mem worker / worker-service / WorkerService greps clean in docker/ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 11 — team-aware generation with audit chain Generation jobs now carry team_id/project_id/api_key_id/actor_id/ source_adapter from enqueue through execution; the outbox is reloaded from Postgres before any side effect so BullMQ payload can never act as auth authority. - src/server/jobs/types.ts — ServerGenerationJobPayloadSchema (Zod discriminated union) requires team_id, project_id, generation_job_id, source_adapter, api_key_id, actor_id (nullable), source_type, source_id, plus event_id / server_session_id per kind. assertServerGenerationJobPayload is called at enqueue (outbox.ts) and again at execution boundary. - src/server/services/{IngestEventsService,EndSessionService}.ts + SessionGenerationPolicy.ts — thread identity context (apiKeyId, actorId, sourceAdapter) into both event and summary BullMQ payloads. - src/server/generation/ProviderObservationGenerator.ts — loadCanonicalOutbox loads the outbox row WITHOUT scope filter, then compares candidate.team_id/project_id to payload.team_id/project_id; mismatch -> ServerGenerationScopeViolationError (non-retryable), failed status, generation_job.scope_violation audit. isApiKeyRevoked checks api_keys (revoked_at, expires_at, row missing) before any provider call; revoked -> generation_job.revoked_key audit + non- retryable failure. generation_job.processing audit emitted on lock. - src/server/generation/processGeneratedResponse.ts — generated observations carry team_id/project_id/server_session_id from the reloaded source row (not job payload). observation_sources.metadata records source_adapter, actor_id, api_key_id for traceability. observation.created audit per observation; generation_job.completed audit per terminal transition. All audit rows reference the same generation_job_id in details. - src/server/routes/v1/ServerV1PostgresRoutes.ts — GET /v1/teams/:id/jobs and GET /v1/projects/:id/jobs with SQL-layer scoping (WHERE team_id=$1 [AND project_id=$2] [AND status=$3]); cross-tenant returns 404 to avoid leaking row existence. Pagination via status/limit/offset. audit_log rows for event.received, event.batch_received, observation.read. - src/server/compat/{SessionsObservationsAdapter,SessionsSummarizeAdapter}.ts — propagate apiKeyId and sourceAdapter='claude-code-compat'. Verification: 162 pass / 10 skip / 0 fail. Pre-existing failures in tests/services/queue and tests/services/worker confirmed independent via git stash. Zero new typecheck errors in server-beta files. Required greps: rg "team_id.*req\.body|project_id.*req\.body" src/server -> 0 matches Audit chain integration test passes — generation_job.processing, observation.created, and generation_job.completed audit rows all share the same generation_job_id reference. Anti-pattern guards verified: - BullMQ payload never acts as auth authority — Postgres outbox reload with mismatch check happens before every side effect - team_id / project_id never derived from request body for scope decisions; always req.authContext.teamId / projectId - Application-layer team/project filtering forbidden — listJobsForScope pushes scope into the SQL WHERE clause - Project-scoped key on cross-project /v1/teams/:id/jobs returns 404 - Revoked api keys cause non-retryable failure with audit before any provider call Deferred: a redundant generation_job.queued audit_log row (already covered by observation_generation_job_events lifecycle log per Phase 1 schema split). Compat adapters set actor_id=null but propagate api_key_id which is the canonical reference downstream. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): Phase 12 — observability and operations Operators can now inspect, retry, and cancel generation jobs from the CLI; queue lane metrics flow into /api/health and /v1/info; every request gets a stable request_id that flows through HTTP -> audit -> outbox -> generator -> completion log. - src/server/middleware/request-id.ts — honors safe inbound X-Request-Id, mints uuid v4 otherwise. Set on req.requestId and echoed via response header so external traces can correlate. - src/server/jobs/ServerJobQueue.ts — QueueEvents wired with completed, failed, progress, stalled, error listeners; lifecycle counters exposed via observe() API. Logs emitted as [generation] job=<id> source_type=<...> duration=<ms> attempts=<N> reason=<message>. Stalled and error counters survive worker restart. - src/server/jobs/types.ts — ServerGenerationJob payload schema extended with optional request_id; flows through from HTTP into every BullMQ job. - src/server/queue/ObservationQueueEngine.ts — health snapshot now carries per-lane (event, summary) counts via ObservationQueueHealthLaneSnapshot. - src/server/runtime/{ActiveServerBetaQueueManager, ActiveServerBetaGenerationWorkerManager,ServerBetaService}.ts — per-lane getJobCounts feed /api/health and /v1/info; stalled events audit through audit_log with action generation_job.stalled. - src/server/routes/v1/ServerV1PostgresRoutes.ts — GET /v1/jobs (status/source_type/since/limit/offset, scope from api-key, payload stripped unless ?include=payload AND admin scope), POST /v1/jobs/:id/retry (idempotent; queued -> no-op; audit generation_job.retried_by_operator), POST /v1/jobs/:id/cancel (terminal -> no-op; audit generation_job.cancelled_by_operator; generator reload-before-side-effects already prevents double work). - src/server/services/IngestEventsService.ts + SessionGenerationPolicy.ts + ProviderObservationGenerator.ts — request_id propagated end to end. Generator extracts request_id from BullMQ payload and includes it in lock/processing/completion logs and audit details. - src/npx-cli/commands/server-jobs.ts + src/npx-cli/commands/server.ts — `claude-mem server jobs status|failed|retry|cancel`. status compares Postgres outbox counts to BullMQ queue counts and surfaces divergence. failed prints attempts + last_error message. --team and --project filters. Verification: 350 pass / 12 skip / 7 fail (pre-existing baseline, verified independent via git stash). 18 new tests added (request-id middleware, server-jobs CLI seams, jobs list/retry/cancel routes Postgres-gated). Zero new typecheck errors. Anti-pattern guards verified: - agent_events.payload only emitted in /v1/jobs response inside the admin-gated branch (?include=payload + admin scope) — returns 403 otherwise - jobs retry on a queued row is a no-op (no double BullMQ enqueue, no double UPDATE) - Every operator action writes to audit_log with the *_by_operator action and request_id correlation in details - Stalled events audit through generation_job.stalled Sample correlated trace (one request_id end to end): HTTP middleware: req.requestId = 'req-abc' audit event.received: details.requestId = 'req-abc' BullMQ payload: { request_id: 'req-abc', generation_job_id: 'gj_x' } generator lock log: [generation] job locked { jobId, requestId } audit generation_job.processing: details.requestId = 'req-abc' completion log: [generation] job=evt_... duration=1230ms Deferred: live /api/health round-trip integration test (needs Redis); stalled event live integration test (needs Redis); storing request_id on the observations row itself (spec did not require). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(server-beta): add Phase 13 release readiness report Captures the final verification gate: tests (1749 pass, 45 fail all pre-existing baseline, zero regressions), required greps clean, Docker E2E green end-to-end, all 7 exit criteria met, build clean, typecheck unchanged from main. Documents deferred items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(server-beta): rebuild server-beta-service bundle Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address Greptile review on PR #2383 - ProviderObservationGenerator.lockOutbox: skip duplicate worker run when another lock is active instead of returning the row, which previously let two BullMQ workers issue the (paid, rate-limited) external provider call before the persistence-layer terminal-status guard collapsed the duplicate. Reconciliation still recovers from a stale lock on startup or next retry. - docker-compose.yml: require POSTGRES_USER/PASSWORD/DB env vars (no defaults). Stack refuses to start without explicit secrets. Added a header warning that the file must not be deployed unmodified. - e2e-server-beta-docker.sh: export ephemeral test creds for the new required env vars so the Docker E2E driver still runs unattended. - ServerBetaService api-key list: bound query with LIMIT/OFFSET (default 100, max 500) and add optional --team filter to prevent unintentional cross-tenant key metadata disclosure on shared admin hosts. - SessionGenerationPolicy: fix dead `??` fallback for NaN parseInt result; use `||` so DEFAULT_DEBOUNCE_MS actually applies. - ServerV1PostgresRoutes: `?wait=true` now actually waits — polls the outbox row until terminal status (timeout 30s, 100ms interval) on both /v1/events and /v1/events/batch. Returns `waitTimedOut: true` if the cap is hit so callers can re-poll the status endpoints. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address CodeRabbit + Greptile second review on PR #2383 P1 fixes - Operator retry endpoint was re-publishing the Postgres outbox metadata column as the BullMQ payload; the worker's assertServerGenerationJobPayload always rejected it, leaving the row stuck in queued until startup reconciliation. Persist the BullMQ payload on the outbox row at create-time inside IngestEventsService and EndSessionService, then re-enqueue that canonical payload on retry. Major fixes - prompt-builder: escape server_session_id when interpolating into the XML prompt; previously a session id containing `<`, `&`, or quotes could inject XML into the provider input. - ServerJobQueue: route both worker.on('stalled') and the QueueEvents 'stalled' subscriber through a single notifyStalled helper that dedupes by jobId for 30s, so counters.stalled increments once per stall. QueueEvents 'error' now routes through notifyQueueError so it increments counters.errored and runs onError listeners — keeping observability symmetric across both sources. - ServerV1PostgresRoutes: convert PostgresObservationRepository from three dynamic imports to a single static import for consistency. - mcp-server / ServerBetaClient: actually forward the observation_record_event tool's `generate` flag through to the /v1/events endpoint as `?generate=false` instead of voiding it. - server-sessions.markGenerationFailed: guard jsonb_set against a null error payload so the failure path can't null out metadata before the generation_status='failed' write commits. Minor fixes - server-sessions.endSession: keep updated_at stable on repeated calls so the documented idempotency contract holds. - SettingsDefaultsManager + ServerBetaService.getServerBetaPort: derive the server-beta default port from UID (37877 + uid%100), matching the worker port pattern, so two users on the same host don't collide. Docker stacks always pass CLAUDE_MEM_SERVER_PORT explicitly so the containerized deployment is unaffected. - server-session-runtime test: close the pg.Pool in afterAll. - server-beta-release-readiness.md: escape pipes inside table inline code, add `text` language tag to the fenced log block. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): address Greptile + CodeRabbit third review on PR #2383 P1 fixes - SessionsObservationsAdapter.resolveServerSession: catch unique-violation (23505) on concurrent compat inserts and re-fetch instead of returning 500. Two compat callers carrying the same contentSessionId can both observe `existing===null` and race on the (project_id, external_session_id) unique constraint; the second now resolves to the raced row instead of dropping the event. - /v1/events/batch: pass `sourceAdapter: null` to ingestBatch so each event's BullMQ payload (and persisted outbox payload column) reflects its own event.sourceAdapter via buildEventBullmqPayload's fallback, rather than stamping the whole batch with the first event's adapter. Minor - server-session-runtime test afterEach: wrap DROP SCHEMA in try/finally so client.release() always runs even if the drop throws. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): drop `pool as never` cast — pg.Pool already matches PostgresPool Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): retry of completed job now 409s instead of duplicating retryGenerationJob previously fell through to the reset+re-enqueue path when called on a job in `completed` status. The observations index dedupes on (generation_job_id, parsed_observation_index, content) but LLM output is non-deterministic, so a second provider run almost always produced a different content string and bypassed the index, persisting a parallel set of observation rows attributed to the same generation job. Match cancelGenerationJob's 409 guard for completed jobs. failed and cancelled remain valid retry targets. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(server-beta): rebuild bundles after rebase onto main Regenerates the three plugin bundles so they reflect the rebased source state. Mechanical rebuild output only — no source changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): wrap resolveServerSession in try/catch for structured error response Greptile P1 on PR #2383: resolveServerSession was called before the try/catch in both compat adapters, so Postgres errors during session lookup (timeout, pool exhaustion, etc.) escaped to Express's default error handler and returned HTML/text 500s. Legacy clients calling response.json() would get a parse failure instead of the documented { stored: false, reason: 'internal_error' } (or { status: 'error', reason: 'internal_error' } for the summarize adapter) shape. Move the resolveServerSession call inside the existing try block in both adapters so any failure flows through the structured catch handler. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): catch 23505 unique violation in POST /v1/sessions/start Greptile P1 on PR #2383: concurrent requests with the same externalSessionId can both pass the findByExternalIdForScope check, both call repo.create, and the loser hits the (project_id, external_session_id) unique constraint. The handler treated that as an unknown error and returned a 500. Apply the same pattern resolveServerSession already uses: catch error.code '23505' when externalSessionId is set, refetch the row inserted by the winning request, and return 200 with that session. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-11 00:26:11 -07:00
Alex Newman	36b0929fae	Server-beta: Postgres storage + independent runtime + BullMQ queue (Phases 1–3) (#2351) ... * Add server beta runtime foundation * Address server beta review findings * Resolve server beta review comments * Tighten server beta review follow-ups * Harden server beta auth and search * Avoid unnecessary FTS rebuilds * Block scoped keys from creating projects * Release BullMQ claims best effort on close * Address server beta review blockers * Reset BullMQ claims best effort * Add Postgres observation storage foundation * feat(server-beta): add independent runtime service Introduce src/server/runtime/ as a self-contained server-beta runtime that owns its lifecycle, Postgres bootstrap, and HTTP boundary without depending on WorkerService. ServerBetaService wraps the existing Server class, exposes /healthz and /v1/info with runtime="server-beta", and persists state to dedicated paths (.server-beta.pid|.port|.runtime.json). The four boundary managers (queue, generation worker, provider registry, event broadcaster) are intentionally disabled in this phase and report their status through /v1/info; later phases activate them. Adds plans/2026-05-07-finish-bullmq-branch-ship-plan.md to track the remaining work for this branch. Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): route CLI lifecycle and bundle separate runtime scripts/build-hooks.js now produces plugin/scripts/server-beta-service.cjs as a separate Node CJS bundle, alongside the existing worker-service bundle. The server-beta runtime is now installable independently. src/npx-cli/commands/server.ts routes start|stop|restart|status to the server-beta lifecycle instead of the legacy worker. The worker keeps its own start|stop|restart|status under the worker namespace; the two runtimes can be operated independently. src/services/worker-service.ts adds a server-* command parser branch that delegates to the sibling server-beta-service.cjs bundle so direct worker-service invocations still route to the right runtime. tests/npx-cli-server-namespace.test.ts updated to expect server-beta lifecycle routing. Includes rebuilt plugin/scripts/*.cjs bundles produced by build-and-sync. Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): add BullMQ job queue primitives Introduce src/server/jobs/ as the queue-side primitives that Phase 3 of the server-beta runtime needs to operate. types.ts defines a discriminated union over the four job kinds (event, event-batch, summary, reindex) and maps each to a per-kind BullMQ queue name and deterministic-ID prefix. job-id.ts builds deterministic, colon-free BullMQ jobIds from (kind, team, project, source). The colon ban exists because BullMQ uses ':' as a Redis key separator internally; embedding ':' in jobIds breaks scan and state lookups. ServerJobQueue.ts is a thin wrapper over BullMQ Queue + Worker that enforces autorun:false, default concurrency 1, and an attached error listener — all per BullMQ docs requirements. Test seams accept queue and worker factories so unit tests do not need Redis. outbox.ts publishes through the Postgres ObservationGenerationJob repository as canonical history. enqueueOutbox writes the row first, then publishes to BullMQ; if BullMQ throws, the row is transitioned to failed and a failed event is appended. reconcileOnStartup re-enqueues queued + processing rows after a restart, replacing terminal BullMQ jobs that may still be holding the deterministic ID slot. markCompleted and markFailed wrap transitionStatus and append the matching event row. Includes 20 unit tests covering deterministic ID stability, colon-free output, queue lifecycle, error-listener attachment, double-start refusal, idempotent enqueue, BullMQ failure rollback, startup reconciliation, max-attempts skipping, and completion / failure / retry transitions. Phase 3 commit 1 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): activate queue boundary in runtime service Wire ActiveServerBetaQueueManager into the server-beta runtime graph. The active manager owns one ServerJobQueue per generation kind (event, event-batch, summary, reindex) and surfaces lane metadata through boundary health. Selection is opt-in and fail-fast: if CLAUDE_MEM_QUEUE_ENGINE is set to bullmq the active manager is constructed (and any Redis/config error throws — no silent fallback to SQLite, per Phase 3 anti-pattern guard). For any other engine the disabled boundary remains so worker-era and test setups stay compatible. Widens ServerBetaBoundaryHealth.status to a discriminated union ('disabled' | 'active' | 'errored') with optional details. The disabled adapter still emits status='disabled', which keeps the existing server-beta-service test green. ServerBetaService receives the manager through a new optional queueManager field on CreateServerBetaServiceOptions so test graphs and Phase 4 wiring can inject custom managers. Adds tests/server/runtime/active-queue-manager.test.ts covering bullmq guard, active health shape, per-kind queue access, close behavior, and post-close errored health. Phase 3 commit 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): cap /v1/events/batch at 500 events Prevents unbounded array DoS surface flagged in PR review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-08 01:20:07 -07:00
Alex Newman	65f2fd8cdd	fix: harden startup and schema repair contracts ... Reliability patch covering startup path resolution, install marker compatibility, export CLI request contracts, schema repair safety, hard-stop retry-loop handling, and the PR babysit status helper.	2026-05-06 18:29:26 -07:00
Alex Newman	c4097b4ebb	add Claude SDK gateway installer setup	2026-05-04 20:21:36 -07:00
Alex Newman	a3b161f8c8	chore: bump version to 12.6.0 ... Minor release: 17 issues fixed and 4 foundations introduced via PR #2282. Foundations (new public modules): - F1 spawnHidden wrapper (src/shared/spawn.ts) — windowsHide default - F2 paths namespace (src/shared/paths.ts) — 24 hardcoded sites collapsed, CLAUDE_MEM_DATA_DIR flows 100% of runtime - F3 getUptimeSeconds (src/shared/uptime.ts) — fixes ms-bug at Server.ts:165 - F4 ClassifiedProviderError (src/services/worker/provider-errors.ts) — kind union (transient/unrecoverable/rate_limit/quota_exhausted/auth_invalid), per-provider classifiers, unrecoverablePatterns allowlist deleted User-facing capabilities: - OAuth keychain reader (#2215): readClaudeOAuthToken() reads from macOS keychain, Windows DPAPI, Linux libsecret at worker spawn-time - Quota-aware wall-clock guard (#2234): RateLimitStore with auth-type gate (api_key never aborts; cli/oauth aborts at per-window thresholds); rateLimits surfaced on /api/health - withRetry helper (#2254): honors ClassifiedProviderError.kind, exponential backoff with jitter, request-id capture for dedup logging Bug fixes also landing in 12.6.0: #2188 stdin fallback, #2196 ANTHROPIC_BASE_URL docs, #2220 chroma CPU storm, #2225 opencode _zod.def crash, #2231 SECURITY.md, #2233 parser fence (Part A), #2236 observer visible windows, #2237/#2238 hardcoded paths, #2240 Chroma dedupe, #2242 check-pending-queue endpoints, #2243 scripts/package.json, #2244 summaryStored=null, #2247 Codex task_complete, #2248 Cursor sessions, #2250 health uptime ms, #2253 chroma macOS CPU. Tests: 1454 pass / 77 fail (matches main baseline, zero net regressions). All CI green: build, CodeRabbit (17 rounds resolved), Greptile (clean). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-03 22:48:11 -07:00
Alex Newman	d384d3c595	fix: bug-batch — 17 issues + 4 foundations (chroma, opencode, parser, OAuth, paths, uptime, classification) (#2282) ... * feat: foundations F1-F4 + simple bug fixes Foundations (no consumer adoption yet): - F1 spawnHidden wrapper at src/shared/spawn.ts - F2 paths namespace with 18 accessors + invariant test (tests/shared/paths.test.ts) - F3 getUptimeSeconds at src/shared/uptime.ts - F4 ClassifiedProviderError at src/services/worker/provider-errors.ts + 6 tests Issue fixes (file-isolated, parallel-safe): - #2231: SECURITY.md at repo root for GitHub Security tab - #2240: dedupe observationIds before Chroma sync (ResponseProcessor.ts) - #2247: add task_complete to Codex session-end events - #2243: rsync excludes scripts/package.json + scripts/node_modules Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: validate Claude executable with --version and detect desktop app Extract findClaudeExecutable() into shared utility used by both SDKAgent and KnowledgeAgent (deduplication). Every candidate is now validated with --version (3s timeout). Desktop app executables in AppData/Program Files get an actionable error message directing users to install the CLI via npm. Closes #2222 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use Zod schemas in OpenCode plugin to fix _zod.def crash OpenCode 1.14.x walks arg._zod.def at plugin registration, which crashes on plain JSON Schema objects like {type: "string"}. Replace with z.string().describe() so the Zod internals are present. Closes #2226, #2225, #2154 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: neutralize chroma-mcp CPU storm at the root Two surgical fixes to the chroma backfill path that together cause the sustained 60–80% CPU + orphan accumulation pattern reported across 1. ChromaMcpManager.getSpawnEnv: cap embedding-thread fanout ONNX Runtime / OpenBLAS / MKL all default to cpu_count(), so a 12-core machine spins 12 threads burning embeddings concurrently. The user's getSpawnEnv only handled SSL certs — no thread limits at all. Inject OMP_NUM_THREADS / ONNX_NUM_THREADS / OPENBLAS_NUM_THREADS / MKL_NUM_THREADS defaults of 2 (only if user hasn't pinned them), and ANONYMIZED_TELEMETRY=false to stop background HTTP from the embedding subprocess. Closes the storm at the source. 2. ChromaSync.backfill{Observations,Summaries,Prompts}: per-batch watermark The bump was in a trailing finally block. SIGKILL / OOM / power loss mid-flight skips finally entirely, so the watermark stayed at 0 and the next worker boot re-embedded the entire history (16K obs in #2220's case), which then pegged CPU forever in combination with (1). Move the bump inside the loop so progress is durable per batch. Closes #2214. Verification: - 26/26 chroma tests pass (tests/services/sync, tests/integration/chroma-vector-sync) - Bundle confirms thread caps and per-batch bumps are present - Full suite: 1429 pass / 20 fail — pre-existing failures only, no regression vs v12.4.9 baseline (1429 pass / 27 fail) Closes #2214. Substantially de-amplifies #2220 (the structural Job-Object cleanup is still tracked separately at #2216). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: kill chroma-mcp process tree and limit backfill concurrency Three fixes for orphan chroma-mcp processes and resource exhaustion: 1. killProcessTree() in ChromaMcpManager.stop() tears down the full uvx->uv->python->chroma-mcp spawn chain (pkill -P on POSIX, taskkill /T on Windows) before MCP client.close(). 2. Register chroma process with pgid for supervisor shutdown cascade. 3. backfillAllProjects() now processes max 3 projects concurrently with a re-entrancy guard to prevent overlapping fire-and-forget runs. Closes #2216, advances #2220, #2213 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * build: regenerate plugin artifacts after cherry-picks Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: foundation consumers + Cursor/stdin/queue/docs fixes F1 spawnHidden adoption (#2236): - 8 spawn → spawnHidden conversions across worker-utils, ProcessManager, npx-cli (install/runtime), supervisor/process-registry F3 getUptimeSeconds adoption (#2250): - Server.ts:165 (THE BUG: returned ms) - Server.ts:270, SessionRoutes.ts:326 (4th ms-bug consumer found), DataRoutes.ts:225 (refactor for consistency) #2188 stdin '{}' fallback removal: - Diagnostic logging to <DATA_DIR>/logs/runner-errors.log + CAPTURE_BROKEN marker; exit 0 to preserve Windows Terminal exit-code strategy #2196 ANTHROPIC_BASE_URL docs: - New docs/public/configuration/custom-anthropic-backends.mdx - Note: issue may need separate auto-detect feature; docs document existing plumbing only #2242 check-pending-queue endpoints: - Point at /api/processing-status + /api/processing per DataRoutes.ts; honor CLAUDE_MEM_WORKER_PORT env #2248 Cursor sessions never summarized: - Pulled reporter wbingli's tested fix (commit 46eaba44) - Bug A: cursor adapter now derives transcriptPath from cwd+sessionId - Bug B: parser accepts both line.type and line.role - Bug C: walk backward, prefer non-empty text, fallback to empty - Tests: 10-case regression suite + tests/fixtures/cursor-session.jsonl Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: F2 paths namespace adoption (#2237 + #2238) Replaced 24 hardcoded homedir() + '.claude-mem' sites across 18 source files with paths.<accessor>() calls from src/shared/paths.ts. Accessors used: dataDir, workerPid, settings, database, chroma, combinedCerts, transcriptsConfig, transcriptsState, corpora, supervisorRegistry, envFile, logsDir. Sites converted (file:area): - src/cli/claude-md-commands.ts (database) - src/services/context/ContextConfigLoader.ts (settings) - src/services/infrastructure/ProcessManager.ts (workerPid) - src/services/infrastructure/WorktreeAdoption.ts (settings) - src/services/integrations/CodexCliInstaller.ts (settings) - src/services/sync/ChromaMcpManager.ts (chroma + combinedCerts) - src/services/transcripts/config.ts (transcriptsConfig + transcriptsState) - src/services/worker/ClaudeProvider.ts (envFile) - src/services/worker/GeminiProvider.ts (envFile + 2 more) - src/services/worker/http/routes/DataRoutes.ts (dataDir) - src/services/worker/http/routes/SettingsRoutes.ts (settings + envFile) - src/services/worker/knowledge/CorpusStore.ts (corpora) - src/shared/EnvManager.ts (envFile) - src/supervisor/index.ts (supervisorRegistry) - src/supervisor/process-registry.ts (supervisorRegistry) - src/supervisor/shutdown.ts (supervisorRegistry) - src/utils/claude-md-utils.ts (database) - src/utils/logger.ts (logsDir + settings, lazy to avoid cycle) CLAUDE_MEM_DATA_DIR override now flows through 100% of the worker runtime; no per-file env reads needed. Verification: - Grep guard: zero homedir+'.claude-mem' sites remain in src/ (excluding paths.ts itself and SettingsDefaultsManager.ts) - F2 invariant test: 3/3 pass (60 expects) - Foundation tests: 19/19 pass Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: F4 provider classification + parser fence + OAuth keychain F4 adoption (#2244 + #2254): - Per-provider classifiers: classifyClaudeError, classifyGeminiError, classifyOpenRouterError. Each lives in the provider file. - New retry helper at src/services/worker/retry.ts: withRetry() honors ClassifiedProviderError.kind; retriable=transient/rate_limit (with retryAfterMs); not retriable=unrecoverable/auth_invalid/quota_exhausted. maxRetries=2, perAttemptTimeout=30s, exponential backoff with jitter. - GeminiProvider + OpenRouterProvider fetch calls wrapped with retry. Best-effort request-id capture (x-goog-request-id, x-request-id, x-openrouter-request-id) for dedup logging. - Deleted unrecoverablePatterns allowlist at worker-service.ts:540 area; worker dispatches on err.kind instead. - 28 new classifier tests at tests/worker/provider-classifiers.test.ts: 429-no-Retry-After, 500-with-quota-exceeded, OverloadedError, per-provider auth_invalid signals. #2233 Part A — parser fence handling: - src/sdk/prompts.ts: removed 4 fence markers from XML example blocks. Model now sees plain XML, eliminating the failure-mode that drained quota via repeated retries. - src/sdk/parser.ts: stripCodeFences() at top, called before parseAgentXml. Fence-tolerant regardless of model behavior. - TODO comment references #2233 Part B (tool-use migration as separate scope). - 4 fence-tolerance tests added to tests/sdk/parser.test.ts. #2215 OAuth token keychain: - New src/shared/oauth-token.ts (~360 LOC): readClaudeOAuthToken() reads from platform-native credential stores at worker spawn-time. - macOS: security find-generic-password -s "Claude Code-credentials" - Windows: PowerShell wrapper around CredRead (Win32 Advapi32.dll) - Linux: secret-tool lookup - Fallback: env CLAUDE_CODE_OAUTH_TOKEN with JWT exp claim or sidecar expiresAt validation; refuses stale-token injection. - EnvManager.buildIsolatedEnvWithFreshOAuth() (async) replaces silent process.env copy. Empty injection on absent; marker write on expired. - <DATA_DIR>/oauth-stale.marker surfaces "re-login via Claude Desktop" via existing SessionStart additionalContext mechanism (context.ts). - ClaudeProvider.startSession + KnowledgeAgent.prime/executeQuery now await the async env builder. - 17 oauth-token tests covering decodeJwtExpMs, marker round-trip, env-fallback expiry detection. Verification: - npx tsc --noEmit: only pre-existing bun-types error - bun test (foundations + new): 70 pass, 0 new fails (8 fails are pre-existing parser.test.ts cases unrelated to fence work) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: #2234 quota-aware wall-clock guard New src/services/worker/RateLimitStore.ts (207 LOC) — vendor pattern from meridian/rateLimitStore.ts (MIT, copied not depended). API: - class RateLimitStore: set/get/getAll/getMostRecentByWindow/size/clear, in-memory last-write-wins keyed by rateLimitType. - globalRateLimitStore singleton. - shouldAbortForQuota(authMethod, store, now?) → {abort, reason?, window?} - isApiKeyAuth(authMethod): matches both verbose getAuthMethodDescription strings and concise "api_key". Thresholds (auth-type gated): - api_key: never aborts (user authorized per-call spend). - cli/oauth/subscription: - five_hour utilization >= 0.95 OR resetsAt within 15min (with 0.85 utilization floor to avoid false trip on freshly-reset windows) - seven_day_opus >= 0.93 - seven_day_sonnet >= 0.92 - seven_day >= 0.93 - overage >= 0.95 ClaudeProvider integration (line 198, for-await loop): - Detects message.type === 'system' && subtype === 'rate_limit' - Records rate_limit_info via globalRateLimitStore.set - Calls shouldAbortForQuota(authMethod, globalRateLimitStore) - On abort: session.abortReason = 'quota:<window>', abortController.abort, break out of loop. Worker continues other sessions. Health endpoint (Server.ts:174): - New rateLimits field on /api/health from getMostRecentByWindow(). - Field shape: {five_hour?, seven_day?, seven_day_opus?, seven_day_sonnet?, overage?} each carrying utilization, status, resetsAt, observedAt. Tests (tests/worker/rate-limit-store.test.ts): - 22 cases covering store CRUD, isApiKeyAuth, abort decision matrix. - api_key never aborts at any utilization. - cli aborts at threshold breaches per window. - Reset-grace buffer with utilization floor. Verification: - npx tsc --noEmit: only pre-existing bun error - bun test tests/worker/rate-limit-store.test.ts: 22/22 pass - bun test tests/claude-provider-resume.test.ts: 9/9 pass - bun test tests/server/: 44/44 pass Plugin artifacts regenerated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build: regenerate worker-service.cjs after final build-and-sync Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: align test assertions with F4 classification + timeout Two test fixes for branch-introduced regressions vs main: 1. tests/gemini_provider.test.ts "should throw on other errors": F4's classifyGeminiError replaced upstream Error message with ClassifiedProviderError. Test was pinned to pre-F4 string. Updated assertion to match new "Gemini bad request (status 400)". 2. tests/infrastructure/graceful-shutdown.test.ts: Test pokes real ~/.claude-mem/supervisor.json registry which on a developer machine contains live worker + chroma-mcp PIDs. SIGTERM → wait → SIGKILL cascade takes ~6s end-to-end. Bumped per-test timeout to 15000ms. Underlying shutdown code unchanged. Future cleanup should mock getSupervisor() here. Result: branch failure count == main (77 pre-existing failures). No new regressions from this branch's work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: address 4 Greptile P1/P2 findings on PR #2282 P1 (real bug): clearStaleMarker silently broken in ESM - src/shared/oauth-token.ts:14: add unlinkSync to top-level fs import - src/shared/oauth-token.ts:342: drop inline require('fs'), call unlinkSync directly. ESM has no require, so the previous code threw ReferenceError swallowed by try/catch — making clearStaleMarker a permanent no-op. Stale oauth marker would persist indefinitely after Claude Desktop refreshed the token. P2 (security): execSync shell-string interpolation - src/shared/find-claude-executable.ts:39: execSync(`"${candidate}" --version`) → execFileSync(candidate, ['--version']). Path containing ", ;, & — reachable on Windows via crafted CLAUDE_CODE_PATH in settings.json — would otherwise produce a malformed/exploitable command. P2 (security): PowerShell username injection - src/shared/oauth-token.ts:119: userInfo().username escaped with PS single-quote convention (' → '') before interpolation into `'Claude Code-credentials:${user}'`. Defensive against future Windows versions or domain-joined machines that may permit ' in usernames. P2 (style): Unreachable throw lastError post-loop - src/services/worker/retry.ts:109: explained as the safety net for opts.maxRetries < 0 (pathological input where the loop never executes and lastError is undefined). Annotated with comment + descriptive fallback Error so the dead-looking code is now self-documenting. Verification: - npx tsc --noEmit: clean (only pre-existing bun-types error) - bun test tests/shared/oauth-token.test.ts tests/worker/provider-classifiers.test.ts tests/worker/provider-errors.test.ts: 50 pass / 0 fail Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: tighten SECURITY.md data-flow and audit dates Fixes CodeRabbit comments #3178957249 (Data Storage section overstated "no external transmission" — softened to call out Claude Agent SDK, alternate provider, Chroma MCP, OAuth keychain, and registry fetches) and #3178957250 (Next Scheduled Audit was earlier than Last Updated; bumped Last Updated to 2026-05-03 and audit to 2026-09-16) on PR #2282. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: drop inline require('fs') in paths.ts Fixes CodeRabbit outside-diff comment on src/shared/paths.ts:25-29 from PR #2282 review. resolveDataDir() ran require('fs') inside an ESM module (this file uses import.meta.url and .js imports), which can break in strict ESM environments. readFileSync now imports at the top alongside existsSync/mkdirSync. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: block CLAUDE_CODE_OAUTH_TOKEN from parent env (issue #2215) Fixes CodeRabbit outside-diff comment on src/shared/EnvManager.ts:14-17 from PR #2282 review. The OAuth-token leak fix was bypassed because buildIsolatedEnv() copied every parent env var that wasn't in BLOCKED_ENV_VARS, but CLAUDE_CODE_OAUTH_TOKEN was not blocked. A stale parent token therefore still reached isolatedEnv even when the fresh keychain read returned expired/absent — defeating the fix documented inline at lines 178-183. Adds CLAUDE_CODE_OAUTH_TOKEN to BLOCKED_ENV_VARS and defensively deletes it again at the top of buildIsolatedEnvWithFreshOAuth() so the fresh-spawn-time read is the only path that can populate it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: validate cursor sessionId against path traversal Fixes CodeRabbit comment #3178957252 on PR #2282. The Cursor adapter took sessionId straight from stdin and concatenated it into a join(homedir(), '.cursor', 'projects', ..., sessionId, ...) path. A crafted value containing path separators or '..' segments could escape ~/.cursor/projects, and the later transcript read would then probe arbitrary local files. deriveCursorTranscriptPath() now rejects any sessionId that doesn't match /^[A-Za-z0-9_-]+$/ — Cursor's real session ids are UUID-style identifiers, so the safe whitelist is non-disruptive. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: scope stripCodeFences() to full-wrapper payloads only Fixes CodeRabbit comment #3178957253 on PR #2282. The previous regex greedily removed the first opening and last closing triple-backticks anywhere in the input, which could mangle valid content with internal fenced examples or surrounding prose — and ran before XML parsing so it created false negatives. stripCodeFences() now only strips when the entire payload is a single fenced block (start-to-end, with optional language tag and surrounding whitespace), capturing the inner content. Adds a regression test that feeds prose with internal triple-backtick markers around a real <observation> block and asserts the inner ``` are preserved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: honor abortSignal during retry backoff sleep Fixes CodeRabbit comment #3178957263 on PR #2282. The retry helper used an unconditional `setTimeout` Promise for backoff between attempts, so an external abort that fired during the wait was delayed until the timer completed. The backoff now races setTimeout against opts.abortSignal: if the signal flips, the timer is cleared and the Promise rejects with 'Aborted' immediately. The abort listener is registered with { once: true } and removed when the timer fires to avoid leaks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: abort immediately on provider-side rejected status Fixes CodeRabbit comment #3178957261 on PR #2282. shouldAbortForQuota() only checked utilization thresholds and reset-grace heuristics; a snapshot with status='rejected' (or overageStatus='rejected' on the overage window) but no utilization number could still return { abort: false }, letting the worker keep consuming after the provider had already declared the bucket exhausted. Provider-side rejection is now checked before utilization. When either rejection signal is present the guard returns abort=true with reason "quota:<window> rejected by provider". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: only bump Chroma watermark on confirmed batch writes Fixes CodeRabbit comments #3178957259 (watermark advances on swallowed batch failures) and #3178957260 (backfillInProgress can stick true if init throws) on PR #2282. addDocuments() previously logged and swallowed per-batch failures with a void return type, so all three backfill loops (observations, summaries, prompts) bumped the watermark unconditionally after the call — turning a transient Chroma failure into permanently-skipped records. addDocuments() now returns the count of documents that actually landed (including delete+add reconcile retries), and each loop only advances the watermark when the batch wrote successfully. Failed batches log a debug message and continue so the loop still gets through the rest. backfillAllProjects() now constructs SessionStore and ChromaSync inside a try block so a constructor throw can't leave the static backfillInProgress guard stuck true and silently skip every later backfill. The finally always clears the guard and best-effort closes each resource. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: fall back to pid kill when process group is gone Fixes CodeRabbit outside-diff comment on src/supervisor/shutdown.ts:118-134 from PR #2282 review. signalProcess() returned silently when a pgid was present and process.kill(-pgid, signal) threw ESRCH, never attempting the per-pid signal. With the new chroma registration path that records a pgid alongside the pid, an already-collapsed group could turn shutdown into a no-op even though the root pid was still alive. The POSIX branch now tries -pgid first when present, and on ESRCH falls through to process.kill(pid, signal). Non-ESRCH errors still propagate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: settings path, uptime clamp, fetch timeouts Fixes three smaller CodeRabbit issues on PR #2282: - SettingsRoutes (outside-diff #2282 review on lines 65-79): the parse-error response told users to delete ~/.claude-mem/settings.json even when paths.settings() resolved elsewhere. Now uses the resolved settingsPath variable in the message. - uptime.ts (#3178957264 / lines 2-3): getUptimeSeconds() could return a negative value if startedAtMs was in the future or the system clock moved backward. Clamps with Math.max(0, ...) so health endpoints never see negative seconds. - check-pending-queue.ts (#3178957248 / lines 27-45): checkWorkerHealth, getProcessingStatus and triggerProcessing all called fetch with no timeout, so the script could block forever if the worker accepted the TCP connection but never responded. Wraps each fetch with an AbortController + 10s timeout that throws a clear timeout message. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: walk descendants recursively when killing chroma-mcp tree Fixes CodeRabbit comment #3178957258 on PR #2282. The POSIX teardown in ChromaMcpManager.killProcessTree() relied on `pkill -P <pid>`, which only signals direct children. Under uv, chroma-mcp spawns python as a grandchild — when uv exits and python re-parents to init, pkill -P never reaches it and the descendant survives the "tree kill". killProcessTree() now collects the full descendant set via a recursive `pgrep -P` walk before each signal phase. The walk returns leaves first so signals propagate bottom-up (SIGTERM children before their parents, then again for SIGKILL after the 500ms grace window so any layer that re-parented during teardown still gets cleaned up). pgrep failures (no children, missing binary) return [] so this stays best-effort and falls back to the existing per-pid signal. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: tolerate malformed JSONL lines in transcript-parser Fixes Greptile P1 comment 3178964456 on PR #2282. extractLastMessageFromJsonl previously called JSON.parse(rawLine) with no guard. A truncated/malformed JSONL line — common when a transcript was crashed mid-write or partially flushed — would throw SyntaxError, crash the summarization pipeline for that session, and silently lose all prior valid messages. Fix: wrap JSON.parse in try/catch and skip bad lines. The empty-line guard only catches truly empty strings, not malformed fragments. Regression tests added for two cases: - Mixed valid + truncated lines: returns last valid match. - All lines malformed: returns empty string (no throw). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: classify FK constraint failures BEFORE provider classifier Fixes Greptile P1 comment 3178979583 on PR #2282. The F4 #2244 work introduced a regression: reclassifyAtDispatch always returns a non-null ClassifiedProviderError for known agent types (Claude/Gemini/OpenRouter), so the isFkConstraintFailure branch was dead code. Per-provider classifiers don't recognize "FOREIGN KEY constraint failed", so SQLite FK failures fell through to the default 'transient' kind and would retry indefinitely — restart loop on corrupted session DB state. Old unrecoverablePatterns explicitly listed FK constraint as unrecoverable; restoring that semantic by checking FK FIRST and only deferring to the classifier when not an FK error. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: validate CLAUDE_MEM_WORKER_PORT in check-pending-queue Parse the env var, range-check (1-65535), and fall back to 37777 with a console.warn on invalid input instead of letting a malformed value flow into the URL builder unchecked (CodeRabbit Minor on PR #2282). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: SIGKILL union of pre-TERM and post-wait descendant sets When the chroma-mcp root exits during the SIGTERM grace window, its descendants get re-parented to init and drop out of the post-wait pgrep -P scan. Without including the pre-TERM snapshot, those re-parented PIDs would never receive SIGKILL even though they were definitely children before SIGTERM and may still be alive (CodeRabbit Major on PR #2282). Compute Array.from(new Set([...descendantsBeforeTerm, ...descendantsBeforeKill])) and SIGKILL the union. The two sets typically overlap, so dedupe is required. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: enforce addDocuments return-count in direct sync paths syncObservation/syncSummary/syncUserPrompt now capture the written count from addDocuments() and only bump the watermark when every requested document landed in Chroma. addDocuments() tolerates per-batch failures (returns the actual written count), so the previous unconditional bump was silently marking unsynced rows as synced on transient errors — preventing the next backfill from retrying them (CodeRabbit Major on PR #2282). A partial write now logs a warn with the (requested, written) pair and preserves retryability on the next pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: guard backfill watermark against non-contiguous failures The backfill watermark is a single monotonic id, so it cannot represent sparse success: "synced through 200, gap at 201–250, then 251 onward" would, on restart, skip 201–250 forever because the watermark sat at either 200 or 251 — both lose data (CodeRabbit Major on PR #2282). Add a per-loop hadGap flag to backfillObservations / backfillSummaries / backfillPrompts. Once any batch under-writes, every subsequent batch must also skip the bump, regardless of whether it itself succeeded. Also tighten the failure check from `writtenInBatch <= 0` to `writtenInBatch < batch.length` so partial-batch writes are caught. The watermark stays at the last contiguously-synced position; the next backfill pass retries from there, eventually closing the gap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: clear oauth-stale marker when token is absent When an OAuth token disappears entirely (user logs out, keychain cleared), buildIsolatedEnvWithFreshOAuth's absent branch was leaving any prior stale-marker file in place. The session-start hook would then keep surfacing an "expired token, re-login" warning even though the token is no longer expired — it's gone, and re-login was already done elsewhere or not applicable (CodeRabbit Minor on PR #2282). Call clearStaleMarker() in the absent branch the same way the present branch already does. Add a regression test exercising the full buildIsolatedEnvWithFreshOAuth path: pre-write a marker, force absent via spoofed unsupported platform, assert the marker is gone after. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: skip unknown message.content shapes instead of throwing extractLastMessageFromJsonl already tolerates malformed JSONL lines (JSON.parse failure -> continue), but a valid JSON line whose message.content is an unexpected type (null, number, plain object) was still throwing — contradicting the new tolerance and crashing the entire summary pipeline on a single weird line (CodeRabbit Major + Greptile P1 on PR #2282). Replace the `throw new Error(...)` with `continue` so a single bad content shape skips that line instead of failing the whole transcript read. Forward compat: future content schemas land harmlessly. Add regression tests covering null, number, and plain-object content; each must not throw and must fall back to the most recent valid line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: guard null/primitive entries in message.content array Fixes CodeRabbit comment 3179004190 on PR #2282. The Array.isArray branch previously did `c.type === 'text'` directly, which throws if `c` is null or a primitive — possible in malformed logs. Tightened the filter with a type guard: requires c to be a non-null object with type === 'text' and a string text field. Same defensive class as the malformed-line and unknown-content-shape tolerances. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-03 22:27:07 -07:00
Alex Newman	9e2973059a	UX redesign: installer + provider rename + /learn-codebase + welcome card + SessionStart hint (#2255) ... * feat(ux): claude-mem UX improvements with installer enhancements Squashed PR #2156 commits for clean rebase onto main: - feat(installer): add provider selection, model prompt, worker auto-start - refactor: rename *Agent provider classes to *Provider - feat: add /learn-codebase skill and viewer welcome card - feat(worker): inject welcome hint when project has zero observations - fix(pr-2156): address greptile review comments - fix(pr-2156): address coderabbit review comments - fix(pr-2156): persist CLAUDE_MEM_PROVIDER for non-claude in non-TTY mode - fix(pr-2156): file-backed settings reads in installer + env-first SKILL doc Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build: rebuild plugin artifacts after rebase onto v12.4.7 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(skills): strip claude-mem internals from learn-codebase The learn-codebase skill, install next-step copy, WelcomeCard, and welcome-hint previously walked the primary agent through worker endpoints and synthetic observation payloads. The PostToolUse hook already captures every Read/Edit the agent makes — the agent should have no awareness that the memory layer exists. Collapse the skill to one instruction ("read every source file in full") and rephrase touchpoints to describe only what the user observes (Claude reading files), not what happens behind the scenes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): preflight version mismatch + settings-aware port resolution Two related fixes for build-and-sync's worker restart step: 1. Read CLAUDE_MEM_WORKER_PORT from ~/.claude-mem/settings.json the same way the worker does, instead of computing the default port from the uid alone. Previously, users with a custom port saw a misleading "Worker not running" message because the restart POST hit the wrong port and got ECONNREFUSED. 2. Add a preflight check that aborts the sync when the running worker's reported version does not match the version we are about to build. Claude Code's plugin loader pins the worker to a specific cache version per session, so syncing into a newer cache directory has no effect until the user runs `claude plugin update thedotmack/claude-mem` to bump the pin. The preflight surfaces this explicitly with the exact command to run; --force bypasses it for intentional cases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(learn-codebase): note sed for partial reads of large files Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: strip comments codebase-wide Removed prose comments from all tracked source. Preserved directives (@ts-ignore, eslint-disable, biome-ignore, prettier-ignore, triple-slash references, webpack magic, shebangs). Deleted two tests that asserted on comment text rather than runtime behavior. Net: 401 files, -14,587 / +389 lines, -10.4% bytes. Verified: typecheck passes, build passes, test count unchanged from baseline (22 pre-existing fails, all unrelated). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(installer): move runtime setup into npx, eliminate hook dead air Smart-install ran 3 times during a fresh install — the worst run was silent, fired by Claude Code's Setup hook after `claude plugin install`, producing ~30s of dead air that looked like the plugin was hung. This change makes `npx claude-mem install` the single place heavy work happens, with a visible spinner. Hooks become runtime-only. - New `src/npx-cli/install/setup-runtime.ts` module: ensureBun, ensureUv, installPluginDependencies, read/writeInstallMarker, isInstallCurrent. Marker schema preserved exactly ({version, bun, uv, installedAt}) so ContextBuilder and BranchManager readers keep working. - `npx claude-mem install`: ungated copy/register/enable for every IDE, inserts a "Setting up runtime" task with honest "first install can take ~30s" spinner. The claude-code shell-out to `claude plugin install` is removed — npx already populated everything Claude reads. - New `npx claude-mem repair` command for post-`claude plugin update` recovery, force-reinstalls runtime. - Setup hook now runs `plugin/scripts/version-check.js` (29ms wall) instead of smart-install. Mismatch prints "run: npx claude-mem repair" on stderr. Always exits 0 (non-blocking, per CLAUDE.md exit-code strategy). - SessionStart loses the smart-install entry; 2 hooks remain (worker start, context fetch). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(installer): delete smart-install sources, retarget tests - Delete scripts/smart-install.js + plugin/scripts/smart-install.js (both are source files kept in sync manually; both must go). - Delete tests/smart-install.test.ts (covered surface is gone). - tests/plugin-scripts-line-endings: drop smart-install.js entry. - tests/infrastructure/plugin-distribution: retarget two assertions at version-check.js (the new Setup hook script). - New tests/setup-runtime.test.ts: 9 tests covering marker read/write, isInstallCurrent semantics. Marker schema invariant verified. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(installer): describe npx-driven setup + version-check Setup hook Sweep public docs and architecture notes to reflect the new flow: npx installer does Bun/uv setup with a visible spinner; Setup hook runs sub-100ms version-check.js; users hit `npx claude-mem repair` after a `claude plugin update`. - docs/architecture-overview.md: hook lifecycle table + npx flow paragraph - docs/public/configuration.mdx: tree + hook config example - docs/public/development.mdx: build output line - docs/public/hooks-architecture.mdx: full rewrite of pre-hook section, timing table, performance table - docs/public/architecture/{overview,hooks,worker-service}.mdx: tree comments, JSON config example, Bun requirement section docs/reports/* untouched (historical incident reports). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): mergeSettings writes via USER_SETTINGS_PATH Greptile P1 (#2156): `settingsFilePath()` only resolved `process.env.CLAUDE_MEM_DATA_DIR`, while `getSetting()` reads via `USER_SETTINGS_PATH` which `resolveDataDir()` populates from BOTH the env var AND a `CLAUDE_MEM_DATA_DIR` entry persisted in `~/.claude-mem/settings.json`. Result: a user with the data dir saved in settings.json but not exported in their shell would have provider/model settings silently written to `~/.claude-mem/settings.json` while `getSetting()` read from `/custom/path/settings.json` — read/write split. Drop `settingsFilePath()` and the now-unused `homedir` import; reuse the already-imported `USER_SETTINGS_PATH` constant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): parse --provider, --model, --no-auto-start install flags Greptile P1 (#2156): InstallOptions has fields `provider`, `model`, `noAutoStart`, but the install case in the npx-cli switch only parsed `--ide`. The other three flags were silently dropped — `npx claude-mem install --provider gemini` was a no-op. Extract a `parseInstallOptions(argv)` helper, share it between the bare `npx claude-mem` and `npx claude-mem install` paths, and validate `--provider` against the allowed set. Update help text accordingly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): pipe runtime-setup output, always show IDE multiselect Two issues caught in a docker test of the installer: 1. The bun.sh installer, uv installer, and `bun install` were using stdio: 'inherit', dumping their stdout/stderr through clack's spinner region — visible as raw "downloading uv 0.11.8…" / "Checked 58 installs across 38 packages…" text streaming under the spinner. Switch to stdio: 'pipe' and surface captured stderr only on failure (via a shared describeExecError() helper that includes stdout when stderr is empty). Spinner stays clean on the happy path. 2. promptForIDESelection() silently picked claude-code when no IDEs were detected, never showing the user the multiselect. On a fresh machine with no IDEs present yet (e.g. our docker test container), the user never got to choose. Now: always show the full IDE list when interactive; mark detected ones with [detected] hints and pre-select them; show a warn line if zero are detected explaining they should pick what they plan to use. Non-TTY callers still get the silent claude-code default at the call site (unchanged). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): skip marketplace work for claude-code-only, offer to install Claude Code Two related UX fixes from a docker test: **Delay between "Saved Claude model=…" and "Plugin files copied OK"** After dropping the needsManualInstall gate, every install was unconditionally running `copyPluginToMarketplace` (which copied the entire root node_modules tree — thousands of files, dozens of seconds) and `runNpmInstallInMarketplace` (npm install --production) even when only claude-code was selected. Neither is needed for claude-code: that path uses the plugin cache dir + the installed_plugins.json + enabledPlugins flag, all of which we already write. - Drop `node_modules` from `copyPluginToMarketplace`'s allowed-entries list; the dependency-install task populates it on the destination side anyway. - Re-introduce `needsMarketplace = selectedIDEs.some(id => id !== 'claude-code')` scoped *only* to `copyPluginToMarketplace`, `runNpmInstallInMarketplace`, and the pre-install `shutdownWorkerAndWait` (also pointless for claude-code- only flows since we're not overwriting the worker's running cache dir source). All other tasks (cache copy, register, enable, runtime setup) stay unconditional. **Claude Code missing → silent install of an IDE that isn't there** When the user picked claude-code on a machine without it (e.g. a fresh container), the install completed but `claude` was unavailable and the only hint was a generic warn line. Replace with an explicit pre-flight prompt: Claude Code is not installed. Claude-mem works best in Claude Code, but also works with the IDEs below. ? Install Claude Code now? ◆ Yes — install Claude Code (recommended) ◯ No — pick another IDE below ◯ Cancel installation If the user picks "Yes", run `curl -fsSL https://claude.ai/install.sh | bash` (or the PowerShell equivalent on Windows), then re-detect IDEs and proceed with claude-code pre-selected. If the install fails or the user picks "No", the multiselect still appears with claude-code visible (just unmarked [detected]), so they can opt in or pick another IDE. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): detect Claude Code via `claude` CLI, not ~/.claude dir The directory `~/.claude` can exist (e.g. mounted in Docker, or created by tooling) without Claude Code actually being installed. Detect the `claude` command in PATH instead so the installer correctly offers to install Claude Code when missing. * docs(learn-codebase): add reviewer note explaining the cost tradeoff The skill intentionally reads every file in full to build a cognitive cache that pays off across the rest of the project. Add a brief note so reviewers (human or bot) understand the tradeoff before flagging the unbounded read as a cost issue. * fix: address Greptile P1 feedback on welcome hint and learn-codebase - SearchRoutes: skip welcome hint when caller passes ?full=true so explicit full-context requests aren't intercepted by the hint. - learn-codebase: replace `sed` instruction with the Read tool's offset/limit parameters, since Bash is gated in Claude Code by default. * feat(install): ASCII-animated logo splash on interactive install Plays a ~1s bloom animation of the claude-mem sunburst logomark when the installer starts in an interactive terminal — geometrically rendered via 12 ray curves around a center disc, in the brand orange. The wordmark and tagline type on alongside the final frame. Auto-skipped on non-TTY, in CI, when NO_COLOR or CLAUDE_MEM_NO_BANNER is set, or when the terminal is too narrow. Inspired by ghostty +boo. * feat(banner): replace rotation frames with angular-sector bloom generator Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): replace rotation frames with angular-sector bloom generator Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): three-act choreography renderer with radial gradient and diff redraw Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): update preview script to support small/medium/hero tier selection Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(docker): add COLORTERM=truecolor to test-installer sandbox Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(install): auto-apply PATH for Claude Code with spinner UX The Claude Code install.sh prints a Setup notes block telling users to manually edit "your shell config file" to add ~/.local/bin to PATH — which left fresh installs unable to launch claude from the command line. After a successful install, detect ~/.local/bin/claude on disk and, if the dir is missing from PATH, append the right export line to .zshrc / .bash_profile / .bashrc / fish config (idempotent, marked with a comment). Also updates process.env.PATH for the current install run. Wraps the curl|bash install in a clack spinner (interactive only) so the ~4 minute native-build download doesn't look frozen — output is captured silently and dumped on failure for debuggability. Non-interactive mode keeps inherited stdio for CI logs. Verified end-to-end in the test-installer docker sandbox: spinner animates, .bashrc gets the export, fresh login shell resolves claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): video-frame ASCII renderer with three-act choreography Generator switched from a single Jimp-rendered logo to pre-extracted video frames concatenated with \x01 separators and gzip-deflated, ported from ghostty's boo wire format. Renderer rewritten around three acts (ignite → stagger bloom → text reveal + breathe) with adaptive sizing, radial gradient, and diff-based redraw. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(onboarding): unify install / SessionStart / viewer around one first-success moment Three surfaces now point at the same north-star moment — open the viewer, do anything in Claude Code, watch an observation appear within seconds — with the same verbatim timing and privacy lines, and a single canonical "how it works" explainer instead of three diverging copies. - Canonical explainer at src/services/worker/onboarding-explainer.md served via GET /api/onboarding/explainer; mirrored into plugin/skills/how-it-works/SKILL.md - SessionStart welcome hint rewritten as third-person status (no imperatives Claude tries to execute), pinned with a default-value regression test - Post-install Next Steps reframed as "two paths": passive default + optional /learn-codebase front-load; drops /mem-search and /knowledge-agent from this surface; adds verbatim timing + privacy lines and /how-it-works link - /api/stats response gains firstObservationAt for the viewer stat row - Viewer WelcomeCard branches on observationCount === 0: empty state shows live worker-connection dot + "waiting for activity"; has-data state shows observations · projects · since [date] and two example prompts. v2 dismiss key - jimp added to package.json to fix pre-existing banner-frame build break Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(banner): play unconditionally; only honor CLAUDE_MEM_NO_BANNER The 128-col / TTY / CI / NO_COLOR gates silently swallowed the banner in narrower terminals, CI logs, and any non-TTY pipe — including Docker runs where -it should preserve the experience but column width was the wrong gate. Remove the implicit gates; keep the explicit opt-out only. If a frame wraps in a narrow terminal, that's better than the banner not playing at all. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * revert(banner): restore 15:33 gating logic per user request Reverts eb6fc157. Restores isBannerEnabled to the state at commit 8e448015 (2026-04-30 15:33): TTY check, !CI, !NO_COLOR, !CLAUDE_MEM_NO_BANNER, and cols >= BANNER.width. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(install): wrap remaining slow steps with spinners Each IDE installer (Cursor, Gemini CLI, OpenCode, Windsurf, OpenClaw, Codex CLI, MCP integrations) now runs inside a clack task spinner with per-step progress messages instead of silent dynamic-import + cpSync. Pre-overwrite worker shutdown (up to 10s) and the post-install health probe (up to 3s) also get spinners. Internal console.log/error/warn from each IDE installer is buffered during the spinner; if the install fails, captured output is replayed afterward via log.warn so users can see what broke. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(review): observation count + IDE pre-selection regressions WelcomeCard's "no observations yet" empty state was triggered when a project filter narrowed the feed to zero rows, even with thousands of observations elsewhere. Source the count from global stats.database to match firstObservationAt's scope. Restore initialValues: [] in the IDE multiselect — pre-selecting every detected IDE was the exact regression #2106 was filed for. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): trichotomy worker state + cache fallback for script path ensureWorkerStarted now returns 'ready' | 'warming' | 'dead' instead of boolean. The spawned-but-still-warming case (common in Docker cold starts and slow first-time inits) was being misreported as 'did not start', which contradicted the next-steps panel saying 'still starting up'. Install task message and Next Steps headline now agree on the actual state. Also fixes the actual root cause of 'Worker did not start' on claude-code-only installs: the worker script path was hardcoded to the marketplace dir, which is left empty when no non-claude-code IDE is selected. Now falls back to pluginCacheDirectory(version) when the marketplace copy isn't present. Verified end-to-end in docker/claude-mem with --ide claude-code, --ide cursor, and a fresh container — install task and headline agree on 'Worker ready at http://localhost:<port>' in all cases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: align CLAUDE.md and public docs with current code Sweep across CLAUDE.md and 10 high-traffic docs/public/ MDX files to remove point-in-time references and align with the actual current shape of the codebase. Highlights: - Hardcoded port 37777 → per-user formula (37700 + uid % 100) on the front-door pages (introduction, installation, configuration, architecture/overview, architecture/worker-service, troubleshooting, hooks-architecture, platform-integration). - Default model 'sonnet' → 'claude-haiku-4-5-20251001' (matches SettingsDefaultsManager). - Node 18 → 20 (matches package.json engines). - Lifecycle hook count corrected (5 events). - Removed the nonexistent 'Smart Install' component and pre-built directory tree referencing files that no longer exist (context-hook.ts, save-hook.ts, cleanup-hook.ts, etc.); replaced with the real worker dispatcher shape. - Removed CLAUDE.md '#2101' issue tag (kept the design rationale). - Replaced obsolete hooks.json example with a description of the real bun-runner.js / worker-service.cjs hook event shape. Lower-traffic doc pages still hardcode 37777 — left for a separate global pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(scripts): land strip-comments around real parsers (postcss, remark, parse5) Each language gets a real parser to locate comments, then we splice ranges out of the original source. The library never serializes — that's how remark-stringify produced 243 reformat-noise diffs in the first attempt versus the 21 real strip targets here. JS/TS/JSX -> ts.createSourceFile + getLeadingCommentRanges CSS/SCSS -> postcss.parse + walkComments + node.source offsets MD/MDX -> remark-parse (+ remark-mdx) + AST html / mdx-expression nodes HTML -> parse5 with sourceCodeLocationInfo shell/py -> kept hand-rolled hash stripper (no library worth the dep) Preserves: shebangs, @ts-* directives, eslint-disable, biome-ignore, prettier-ignore, triple-slash refs, webpack magic, /*! license keep, @strip-comments-keep file marker. JS/TS handler runs a parse-roundtrip check and refuses to write if syntax errors increased (catches the worker-utils.ts breakage class from the 2026-04-29 attempt). npm scripts: strip-comments (apply) strip-comments:check (CI-style, exits non-zero if changes needed) strip-comments:dry-run (list, no writes) Verified --check on this repo: 21 changes, -4.0% bytes, no parse-error regressions, no reformat-suspect false positives. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: strip comments codebase-wide via parser-backed tool 21 files changed, -17,550 bytes (-4.0%) of narrative comments removed across .ts / .tsx / .js / .mjs and the .gitignore. JS/TS comments stripped via ts.createSourceFile + getLeadingCommentRanges — same canonical lexer, same behavior as the 2026-04-29 strip, no reformat noise. Preexisting baseline (unchanged): typecheck: 16 errors at HEAD, 16 errors after strip (line numbers shift, no new error classes — verified via diff of sorted error lists) build: fails at HEAD with CrushHooksInstaller.js unresolved import (preexisting, unrelated to this strip) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): drop Crush integration references after extract The Crush integration was extracted to its own branch on May 1, but the import at install.ts:280 (and the case block + ide-detection entry + McpIntegrations config + npx-cli help text) still referenced the now- removed CrushHooksInstaller.js, breaking the build. Removes: - case 'crush' block in install.ts - crush entry in ide-detection.ts - CRUSH_CONFIG and registration in McpIntegrations.ts - 'crush' from the IDE Identifiers help line in index.ts Rebuilds worker-service.cjs to match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(banner): mark generated banner-frames.ts with @strip-comments-keep Without this, every build/strip cycle ping-pongs five lines of doc comments in and out of the auto-generated output. The keep-marker tells strip-comments.ts to skip the file entirely. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(build): drop banner-frame regen from build script generate-banner-frames.mjs requires PNG frames in /tmp/cmem-banner-frames that only exist after the maintainer runs ffmpeg locally on the source video. CI has neither the video nor the frames, so the build broke on Windows. The output (src/npx-cli/banner-frames.ts) is committed, so the regen is a one-shot dev step — not a build step. Run the script directly when the video changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(worker): unstick the spinner — kill claim-self-lock, wake on fail, auto-broadcast Three surgical changes that cure the stuck-spinner bug at the source. Phase 1.1 (L9): claimNextMessage no longer self-excludes its own worker_pid. A single UPDATE-RETURNING grabs the oldest pending row by id. Removes the LiveWorkerPidsProvider plumbing that was never injected — Supervisor enforces single-worker via PID file, so the multi-worker SQL was defending against a configuration the project does not support. Phase 1.2 (L19): SessionManager.markMessageFailed wraps PendingMessageStore.markFailed and emits 'message' on the per-session EventEmitter. The iterator's waitForMessage now wakes immediately on re-pend instead of parking for 3 minutes. ResponseProcessor and SessionRoutes routed through the new wrapper. Phase 1.3 (L24): PendingMessageStore takes an optional onMutate callback fired from every mutator (enqueue, claimNextMessage, confirmProcessed, markFailed, transitionMessagesTo, clearFailedOlderThan). SessionManager wires it; WorkerService passes broadcastProcessingStatus. Ten manual broadcast calls deleted across SessionCleanupHelper, SessionEventBroadcaster, SessionRoutes, DataRoutes, and worker-service. Caller discipline becomes structurally impossible to forget. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(worker): delete dead code — legacy routes, processPendingQueues, decorative guards Pure deletions. Phase 2 of kill-the-asshole-gates. - Legacy /sessions/:sessionDbId/* routes (handleSessionInit, handleObservations, handleSummarize, handleSessionStatus, handleSessionDelete, handleSessionComplete) bypassed all five ingest gates and were a parallel write path. Folded the initializeSession + broadcastNewPrompt + syncUserPrompt + ensureGeneratorRunning + broadcastSessionStarted work into the canonical /api/sessions/init handler so the hook makes one round trip instead of two. - processPendingQueues (~104 lines, zero callers) — replaced in Phase 6 by a one-statement startup sweep. - spawnInProgress Map and crashRecoveryScheduled Set — decorative dedupe over generatorPromise and stillExists checks that already provide the real safety. - STALE_GENERATOR_THRESHOLD_MS — pre-empted live generators and raced with the finally block; the 3min idle timeout already kills zombies. - MAX_SESSION_WALL_CLOCK_MS — ran a SELECT on every observation to enforce 24h. Runaway-spend protection lives in the API key, not in claude-mem. - Missing-id 400 in shared.ts ingestObservation — Zod already enforces min(1) on contentSessionId and toolName at the route schema. - SessionCompletionHandler import + completionHandler field on SessionRoutes (orphaned after handler deletions). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(worker): SQL-backed getTotalQueueDepth — single source of truth Was: iterate this.sessions.values() and sum getPendingCount per session. Now: SELECT COUNT(*) FROM pending_messages WHERE status IN ('pending','processing'). The in-memory sessions Map drifted from the DB rows whenever a generator exited without confirm/fail, leading to false-positive isProcessing in the UI. Phase 1.3's auto-broadcast fires on every mutation, but it broadcast a stale Map count. Reading from the DB makes the UI's spinner state match what the queue actually holds. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(worker): typed abortReason replaces wasAborted boolean Was: a boolean wasAborted that lumped every abort together. The finally block branched on !wasAborted, so any abort skipped restart — including idle aborts with pending work, which is exactly the case where we DO want to restart. Now: ActiveSession.abortReason is a typed enum 'idle' | 'shutdown' | 'overflow' | 'restart-guard'. The finally block consumes the reason and only skips restart for 'shutdown' and 'restart-guard'. Idle and overflow aborts fall through, so if pending work exists they trigger restart correctly. Dropped 'stale' and 'wall-clock' from the union — Phase 2 deleted those paths. Natural-completion abort (post-success) intentionally has no reason; it's not gating restart logic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(worker): unify the two generator-exit finally blocks Was: worker-service.ts:startSessionProcessor and SessionRoutes:ensureGeneratorRunning each had their own ~70-line finally block with divergent restart-guard handling. The worker-service path called terminateSession on RestartGuard trip and orphaned pending rows (the L16 bug); the SessionRoutes path drained them. Two places to update when rules changed. Now: handleGeneratorExit in src/services/worker/session/GeneratorExitHandler.ts owns the contract: 1. Always kill the SDK subprocess if alive. 2. Always drain processingMessageIds via sessionManager.markMessageFailed (which wakes the iterator — Phase 1.2). 3. shutdown / restart-guard reasons: drain pending rows via transitionMessagesTo('failed'), finalize, remove from Map. Fixes L16. 4. pendingCount=0: finalize normally and remove from Map. 5. pendingCount>0: backoff respawn via per-session respawnTimer (no global Set; Phase 2.4 deleted that). RestartGuard trip drains to 'abandoned'. Both finally blocks are now ~10-line wrappers that translate local state into the canonical abortReason and delegate. Restored completionHandler injection into SessionRoutes (was dropped in Phase 2 cleanup; needed by the unified helper for finalizeSession). Behavior change: SessionRoutes' previous "keep idle session in memory" was deliberately replaced by the plan's "remove from Map on natural completion" — next observation reinitializes via getMessageIterator → initializeSession. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(worker): startup orphan sweep — reset 'processing' rows at boot When the worker dies (crash, kill, restart), any pending_messages rows it left in 'processing' state are by definition orphans (the only worker is dead). Single SQL UPDATE at boot resets them to 'pending' so the iterator can claim them again. Replaces the deleted processPendingQueues function (Phase 2.2). Runs in initializeBackground after dbManager.initialize() and before the initializationComplete middleware releases blocked HTTP requests, so no in-flight request can race the sweep. NOT on a periodic timer — after boot, every 'processing' row has a live consumer and a periodic sweep would race. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(worker): simplify enqueue catch, replace memorySessionId throw with re-pend 7.1: queueObservation's catch was logging two ERROR-level messages and rethrowing. The rethrow is correct (FK violations / disk full / schema drift should crash loudly), but the verbose ERROR logging pretended the error was recoverable. Reduced to one INFO line + rethrow. 7.2: ResponseProcessor's memorySessionId guard was throwing if the SDK hadn't included session_id on the first user-yield, terminal-failing the entire batch. Now warns and re-pends in-flight messages via sessionManager.markMessageFailed (which wakes the iterator — Phase 1.2). The next iteration tries again with memorySessionId hopefully captured. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): mirror builds to installed-version cache for hot reload When package.json bumps past Claude Code's installed pin, sync-marketplace wrote new code to cache/<buildVersion>/ but the worker loaded from cache/<installedVersion>/, so worker:restart reloaded the same old code. Replace the exit-on-mismatch preflight with a mirror step: when versions differ, also rsync plugin/ into cache/<installedVersion>/ so worker:restart hot-reloads new code without a Claude Code session restart. The build-version cache still gets written for the eventual `claude plugin update`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: delete dead barrel files and orphan utilities - src/sdk/index.ts (re-exports parser+prompts; nothing imported the barrel) - src/services/Context.ts (re-exports ./context/index.js; no importers) - src/services/integrations/index.ts (no importers) - src/services/worker/Search.ts (3-line barrel of ./search/index.js) - src/services/infrastructure/index.ts: drop CleanupV12_4_3 re-export - src/utils/error-messages.ts (getWorkerRestartInstructions never imported) - src/types/transcript.ts (170 LoC of types, zero importers) - src/npx-cli/_preview.ts (banner dev preview, no script wires it) Build + tests still pass; observations still flowing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(parser): drop unused detectLanguage Only the user-grammar-aware variant detectLanguageWithUserGrammars() is actually called. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(types): drop unused SdkSessionRecord + ObservationWithContext Both interfaces in src/types/database.ts had zero importers anywhere in src or tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(npx-cli): drop unused getDetectedIDEs + claudeMemDataDirectory getDetectedIDEs has no callers — install.ts uses detectInstalledIDEs directly. claudeMemDataDirectory has no callers either. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(ProcessManager): drop dead orphan-reaper + signal-handler helpers Each had zero callers in src/ or tests/: - cleanupOrphanedProcesses + enumerateOrphanedProcesses - ORPHAN_PROCESS_PATTERNS + ORPHAN_MAX_AGE_MINUTES - forceKillProcess - waitForProcessesExit - createSignalHandler - resetWorkerRuntimePathCache The orphan reaper was retired in PATHFINDER Plan 02 ("OS process groups replace hand-rolled reapers", commit 94d592f2) — these were the leftover pieces. shutdown.ts uses the supervisor's own kill-pgid path instead. parseElapsedTime kept (covered by tests/infrastructure/process-manager.test.ts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(scripts): delete 11 unreferenced DX/forensic scripts None of these are referenced by package.json npm scripts or docs/. All last touched on Apr 29 only as part of the comment-stripping pass — the feature code itself is older and orphaned: analyze-transformations-smart.js debug-transcript-structure.ts dump-transcript-readable.ts endless-mode-token-calculator.js extract-prompts-to-yaml.cjs extract-rich-context-examples.ts find-silent-failures.sh fix-all-timestamps.ts format-transcript-context.ts test-transcript-parser.ts transcript-to-markdown.ts These are standalone tools — runtime behavior unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(scripts): delete unused extraction/ and types/ subdirs - scripts/extraction/{extract-all-xml.py, filter-actual-xml.py, README.md} point at ~/Scripts/claude-mem/ — the user's pre-relocation path that no longer exists. Zero references in package.json, src/, or tests/. - scripts/types/export.ts duplicates ObservationRecord etc. and has no importers (CodexCliInstaller imports transcripts/types, not this). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(BranchManager): drop dead getInstalledPluginPath OpenCodeInstaller has its own (used) getInstalledPluginPath; the BranchManager copy never had any external callers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(ChromaSyncState): unexport DocKind (used internally only) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(gemini): drop stale earliestPendingTimestamp / processingMessageIds Both fields were removed from ActiveSession in earlier queue-engine cleanup. Tests had been silently keeping them because the mock sessions use 'as any' to bypass strict typing, so the dead fields rode along without complaint. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: drop 3 unused module-level constants - src/npx-cli/banner.ts: CURSOR_HOME, CLEAR_DOWN (banner uses CLEAR_SCREEN which combines clear-down + cursor-home into a single CSI sequence; the standalone constants were leftovers). - src/services/worker/BranchManager.ts: DEFAULT_SHELL_TIMEOUT_MS (BranchManager only uses GIT_COMMAND_TIMEOUT_MS / NPM_INSTALL_TIMEOUT_MS). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(opencode-plugin): drop dead workerPost helper Only the fire-and-forget variant (workerPostFireAndForget) is actually called. workerPost was the await-result version with no remaining caller. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: drop 8 truly-unused interface fields Verified each by grepping for `.field`, `"field"`, `'field'`, and `field:` patterns across src/ + tests/ + plugin/scripts. Where the only remaining usage was the assignment site, removed the assignments too. - GitHubStarsData: watchers_count, forks_count (only stargazers_count read) - TableColumnInfo: dflt_value (PRAGMA returns it but no caller reads it) - IndexInfo: seq (PRAGMA returns it but no caller reads it) - ObservationRecord: source_files (legacy field, no readers) - HookResult.hookSpecificOutput: permissionDecisionReason - WatchTarget: rescanIntervalMs (set in config, never read) - ShutdownResult: confirmedStopped (write-only — assigned but no reader; updated all 3 return sites to drop it) - ModePrompts: language_instruction (multilingual support never wired) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(npx-cli): reuse InstallOptions type instead of inline duplicate parseInstallOptions had its return type written out inline as an anonymous duplicate of InstallOptions. Use the canonical type (import type — zero bundle cost). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(integrations): drop unused Platform type alias The detectPlatform() function that returned this type was deleted earlier in the branch (along with getScriptExtension that consumed it). The type itself outlived its consumer; only string literals "Platform:" survive in console.log diagnostics, which don't reference the alias. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(worker): broadcast processing_status when summarize is queued broadcastSummarizeQueued was an empty no-op even though handleSummarizeByClaudeId calls it after enqueueing. The PendingMessageStore onMutate callback already fires broadcastProcessingStatus on enqueue, but calling it explicitly from broadcastSummarizeQueued ensures the spinner ticks on the moment a summary is requested even if the onMutate chain has any timing race. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(worker): keep spinner on while summary generates ClaudeProvider's SDK can pull multiple synthetic prompts (e.g. observation + summarize) before producing responses. Each pull pushed an ID to session.processingMessageIds. When the SDK's first observation response came back, ResponseProcessor.confirmProcessed deleted ALL pending message rows — including the still-in-flight summary — so getTotalQueueDepth dropped to 0 and the spinner turned off, even though the summary took another ~22s to actually generate. Tag each in-flight message with its type ({id, type}) so the response processor can pop only the FIFO message of the matching type (observation vs summarize). The summary row stays in 'processing' until its own response arrives, keeping the spinner lit through the entire summary window. Also updates Gemini/OpenRouter providers and GeneratorExitHandler for the new shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(worker): clear summary from queue on any SDK response Switch ResponseProcessor from type-aware FIFO matching to strict FIFO popping (each SDK response → 1 in-flight message consumed). This way the summary always clears when the SDK responds, even when the response is unparseable or the summary doesn't actually generate content — preventing stuck spinner / queue-depth-stuck-at-1. Spinner behavior is preserved: messages enqueued after the summary keep the queue depth elevated, and only when the SDK has responded to every prompt does the queue drain to zero. Also: when the consumed message is a 'summarize' and parsing fails, treat it as best-effort and confirmProcessed (no retry) — summaries that can't be parsed shouldn't keep retrying. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(viewer): redesign welcome card and remove source filters The first-start welcome card now explains the three feed card types (observation/summary/prompt) with color-coded badges, points users at the gear icon for settings and the project dropdown for filtering, and plugs /mem-search for recall — replacing the old two-line "ask:" prompts. Source filter tabs (Claude/Codex/etc.) are removed from the header. Filtering by AI provider was nonsense from a user POV; the project dropdown is the only header filter now. Source tracking is also stripped from useSSE, usePagination, App state, and CSS. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(viewer): keep welcome card in feed column, swap rows for 3 squares Two visible problems in the previous design: the card stretched edge-to-edge while feed cards sit in a centered 650px column, and the body was a stack of long horizontal rows that scanned line-by-line. Both fixed: Feed now accepts a pinnedTop slot so the welcome card renders inside the same .feed-content column as observation cards. Body is now a 3-column grid of square feature blocks — Live feed, Tune it, Recall it — each with a custom inline SVG illustration (stacked cards with color-coded stripes, gear+sliders, magnifier over cards). Old text-row sections (welcome-card-types, welcome-card-tips, welcome-card-section, welcome-card-tip-icon) are removed. Squares stack to one column under 600px. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(viewer): convert welcome card to glassy modal with stylized logo Card now opens as a centered modal with a frosted/glass backdrop (blur + saturate) so it doubles as a proper help dialog when reopened from the header's question-mark button. Removed the observation count, project count, and "since" date — those don't make sense for a first-launch surface and felt out of place in a help context. Header art swapped from the small webp logomark to the new high-resolution sun/sunburst PNG (claude-mem-logo-stylized.png), shipped as a checked-in asset in src/ui and plugin/ui. Bigger throughout: 28px h2, 16px tagline, 88px illustrations, 26px feature padding, 1:1 aspect-ratio squares. Backdrop click and Esc both close. Mobile collapses the grid to one column and drops the aspect-ratio constraint. Reverted the unused pinnedTop slot on Feed.tsx since the welcome card is now a true overlay rather than an in-feed pinned card. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(viewer): make welcome modal actually glassy Previous version had a 55%-opacity black backdrop that almost fully blocked the underlying UI — the "glass" was just a dark plate. Now the backdrop is fully transparent (no darkening at all), the panel itself drops to 55% bg-card opacity with its existing backdrop-filter blur(28px) saturate(170%), and the feature squares drop to 35% bg-tertiary so they layer as glass-on-glass over the already-blurred panel. The header and feed below now read clearly through the modal's frosted blur. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(viewer): bulletproof square features via padding-bottom + clamp() fluid type Squares were rendering taller than wide because aspect-ratio is treated as a minimum — content can push the box past 1:1. Switched to the classic padding-bottom: 100% trick: percentage padding resolves against the parent's width, so the box is ALWAYS W × W regardless of content. Inner content sits in an absolutely-positioned flex column that can't push the shell taller. Whole modal is now desktop-first and fluid via clamp() — no media-query stair-steps for type, padding, gaps, border-radius, illustration size, or modal width. Single mobile breakpoint at <600px collapses the grid to one column and reverts the padding-bottom trick so each feature can grow to natural content height. Tightened the three feature descriptions so they fit comfortably inside the square at the desktop size. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(viewer): 15% black overlay + heavier modal shadow for elevation Backdrop goes from transparent to rgba(0,0,0,0.15) — just enough darkening to push the modal visually forward without burying the underlying UI. Modal shadow stacked: 40px/120px ambient + 16px/48px contact, both deeper, plus the existing inset 1px highlight. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(build): clear pending_messages queue on build-and-sync Rewrites scripts/clear-failed-queue.ts to talk directly to SQLite via bun:sqlite — the previous HTTP endpoints (/api/pending-queue/*) were removed during the queue engine rewrite, so the script was orphaned. Wires `npm run queue:clear` into `build-and-sync` so each rebuild starts with a clean queue. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(worker): collapse parser to binary valid/invalid + clearPendingForSession model - Parser: { valid: true, observations, summary } | { valid: false } — drops kind/skipped enum dispatch - ResponseProcessor: two branches only (parseable → store + clearPendingForSession; else → no-op) - Drop processingMessageIds + per-message claim/confirm/markFailed lifecycle across 3 providers - PendingMessageStore: 226 → 140 lines; remove markFailed/transitionMessagesTo/confirmProcessed/clearFailedOlderThan/getAllPending/peekPendingTypes... wait keep peekPendingTypes - Schema migration v31+v32: drop retry_count, failed_at_epoch, completed_at_epoch, worker_pid columns - SessionQueueProcessor: delete two 1s recovery sleeps (let iterator end on error) - Server.ts/SettingsRoutes.ts: replace four magic-number setTimeout exit-flush patterns with flushResponseThen helper - GeneratorExitHandler: 183 → 117 lines (drain in-flight loop gone) Net: -181 lines. No more silent data loss via maxRetries=3. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pr-2255): address review comments batch 1 - install.ts: needsMarketplace true when claude-code selected (P1, was no-op) - install.ts: throw on invalid --model so CLI exits non-zero - install.ts: skip worker health checks + adapt next-step copy when --no-auto-start - install.ts: repair regenerates plugin cache when missing - index.ts: readFlag rejects missing/flag-shaped values - index.ts: route flag-first invocations (e.g. `--provider claude`) to install - banner.ts: fail-open if frame payload decode throws - SearchRoutes.ts: 5s TTL cache for settings reads on hot hook path (P2) - detect-error-handling-antipatterns.ts: trailing-brace strip whitespace-tolerant - investigate-timestamps.ts: compute Dec 2025 epochs at runtime (was Dec 2024) - regenerate-claude-md.ts: include workingDir in fallback walker so root is covered - sync-marketplace.cjs: parseWorkerPort validates 1..65535 before http.request - sync-to-marketplace.sh: resolve SOURCE_DIR from script location, not cwd - Dockerfile.test-installer: bash --login sources .bashrc via .bash_profile - docs/configuration.mdx: drop nonexistent .worker.port file refs, use settings.json - docs/architecture-overview.md: dynamic port + queue model after parser collapse - docs/architecture/worker-service.mdx: dynamic port example + drop port-file claim - docs/platform-integration.mdx: WORKER_BASE_URL pattern, drop hardcoded 37777 - install/public/install.sh: Node 20 floor (was 18) to match docs Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pr-2255): reset claimed messages to pending on early-return paths ResponseProcessor returns early in two cases: - parser invalid (unparseable response) - memorySessionId not yet captured Both paths previously left the just-claimed message in `status='processing'`, which counts toward `getPendingCount`. The generator-exit handler then sees `pendingCount > 0` and respawns the generator, looping until the restart guard trips and `clearPendingForSession` deletes the message — silent data loss. Calling `resetProcessingToPending` on these paths lets the next generator pass re-claim the message and try again, instead of burning the restart budget on no-op respawns. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pr-2255): swebench fallback row + troubleshooting port path - evals/swebench/run-batch.py: append fallback prediction row when orchestrator future raises, preserving "never drop an instance" guarantee - docs/troubleshooting.mdx: drop nonexistent .worker.port / worker.port file references; use settings.json + /api/health for port discovery Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pr-2255): memoize per-project observation count for welcome-hint hot path handleContextInject runs on every PostToolUse hook (after every Read/Edit). The welcome-hint block ran a COUNT(*) on observations for every call once CLAUDE_MEM_WELCOME_HINT_ENABLED was true. Observation counts are monotonically increasing — once a project has any observations it always will — so cache the positive result in a Set and skip the COUNT(*) on subsequent requests. Combined with the 5s settings TTL added earlier, the steady-state cost on the hook hot path drops to a Set lookup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pr-2255): use clearProcessingForSession on AI-success path clearPendingForSession deletes ALL rows for the session. On the success path of processAgentResponse, that's wrong: messages that arrived as 'pending' during the (1-5s) AI response latency get deleted along with the 'processing' row we just consumed. In a hook burst (three quick PostToolUse hooks), B and C land while A is in flight; A's success then nukes B and C — silent data loss. Add a status-scoped clearProcessingForSession to PendingMessageStore + SessionManager, and use it in ResponseProcessor's success path. The unconditional clearPendingForSession remains correct in GeneratorExitHandler for hard-stop / restart-guard-trip paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Revert "fix(pr-2255): use clearProcessingForSession on AI-success path" This reverts commit a08995299c30cbad36bddc3e5bddda7af8604b35. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-02 16:05:56 -07:00
Alex Newman	46d204ee9b	Integration: 7 critical fixes (post band-aid strip) (#2219) ... * fix: strip privacy tags from last_assistant_message in summarize path (cherry picked from commit bd68bfcc3cfe9d82977d5bdb87cf7e91a7258489) * fix: preserve Chroma relevance ordering in SQLite hydration When ChromaSearchStrategy queries by vector similarity with orderBy='relevance', SessionStore.getObservationsByIds and related methods silently coerced undefined to 'date_desc', destroying the semantic ranking. Add 'relevance' as a valid orderBy value that skips SQL ORDER BY and preserves caller-provided ID order. Fixes #2153 (cherry picked from commit 9fedf8fc165c01cc3a8a8cdb8c057ea980bf511e) * test(privacy): mock executeWithWorkerFallback and loadFromFileOnce Update the cherry-picked privacy-tag stripping test from swithek's fork to match current main: - Mock executeWithWorkerFallback / isWorkerFallback (the handler now uses these instead of workerHttpRequest directly). - Mock loadFromFileOnce in hook-settings.js (called by shouldTrackProject) so the handler resolves CLAUDE_MEM_EXCLUDED_PROJECTS to a string. - Switch the workerCallLog shape to record { path, method, body } and accept either object or JSON-string bodies. 10/10 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: pass relevance through to SessionStore in ChromaSearchStrategy The Chroma strategy was coercing orderBy='relevance' to undefined before calling SessionStore. Combined with SessionStore's date_desc default for undefined, this destroyed the semantic ranking that Chroma had just computed. Pair this with the SessionStore-side fix from rogerdigital (commit 37c8988f) which now accepts 'relevance' as a valid orderBy and preserves caller-provided ID order. Adds a regression test asserting that getObservationsByIds returns rows in caller-provided order when orderBy='relevance', and continues to return date_desc order when orderBy is omitted. Closes #2153 Co-Authored-By: Roger Deng <13251150+rogerdigital@users.noreply.github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: isolate SDK boundary — settingSources, strictMcpConfig, cloud-provider env, observation cap Single architectural fix at the three @anthropic-ai/claude-agent-sdk query() call sites (SDKAgent.startSession, KnowledgeAgent.prime, KnowledgeAgent .executeQuery) plus the env sanitizer and ingest gate. Closes 6 issues: - #2155 settings.json bleed-through into observer SDK subprocess: pass settingSources: [] so user/project/local settings aren't inherited. - #2159 / #2171 / #2194 user MCP servers leak into observer SDK: pass strictMcpConfig: true alongside the existing mcpServers: {}. - #2199 Bedrock/Vertex env vars dropped: extend ENV_PRESERVE in src/supervisor/env-sanitizer.ts to keep CLAUDE_CODE_USE_BEDROCK, CLAUDE_CODE_USE_VERTEX, AWS_*, ANTHROPIC_VERTEX_PROJECT_ID, etc. - #2201 runaway tokens (345M/day reported): extend default CLAUDE_MEM_SKIP_TOOLS with exec_command, write_stdin, apply_patch and add a configurable CLAUDE_MEM_MAX_OBSERVATION_BYTES (default 64 KB) cap at the ingest gate. SDK option names verified against node_modules/@anthropic-ai/claude-agent-sdk/sdk.d.ts: settingSources?: SettingSource[] (SettingSource = 'user'|'project'|'local') strictMcpConfig?: boolean Anti-pattern guards observed: - Did not modify the proxy strip (#2099/#2115). - Did not skip Read/Write/Edit/Bash — those remain the primary observation surface; only added high-volume agentic-tool names (exec_command, write_stdin, apply_patch). - Did not invent SDK options. Closes #2155, #2159, #2171, #2194, #2199, #2201 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: restore Windows spawn fix from PR #751 + add Windows CI Re-applies the PowerShell Start-Process -WindowStyle Hidden daemon spawn that PR #751 (e6ae0176) introduced and commit d13662d5 reverted. Also fixes the bun-runner cmd /c popup, sets detached:false on Windows for SDK subprocesses (so windowsHide actually works and claude.exe doesn't outlive the worker), and adds windows-latest CI to prevent regression. - ProcessManager.spawnDaemon: PowerShell -EncodedCommand branch back. Returns 0 sentinel on success — callers MUST use pid === undefined for failure detection, never falsy checks. - bun-runner.js: drop "cmd /c" wrapper. shell:true lets Node resolve bun.cmd via PATHEXT and respects windowsHide (the explicit cmd.exe wrapper was popping a visible window per hook — #2150, #2186). - process-registry.ts spawnSdkProcess: detached:false on Windows. Mixing detached:true with windowsHide:true is documented-undefined on Windows; with detached:false, windowsHide actually hides claude.exe and the SDK subprocess dies with the parent (#2190, #2198). - .github/workflows/windows.yml: smoke test counts visible cmd windows before/after spawn + grep guard that the Start-Process branch survives. WSL bash stdin (#2188) is acknowledged but deferred — the bash → node pipe boundary needs a real Windows VM to test, beyond this PR's scope. PTY for Claude CLI SDK mode (#2173, #2177) is also deferred per plan. Closes #2150, #2169, #2186, #2187, #2190, #2198 Refs #2183 (Windows perf — same root cause) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: Codex transcript ingestion + queue self-deadlock on Windows Three Windows-specific bugs surfaced by @MakaveliGER in #2192: A. Glob path normalization path.join(homedir(), ...) emits backslashes on Windows. globSync treats backslashes as escape characters, not separators, so it silently fails to match transcript files. Normalize backslashes to forward slashes before passing to globSync (only affects Windows; Unix paths unchanged). B. Live appends not picked up Per-file fs.watch on Windows ReFS/SMB misses appends to live JSONL files; the recursive root watcher is the only signal we can trust there. Expose FileTailer.poke() and call it from the root-watcher event when the file is already tailed, instead of returning early. Also normalize the resolved path so the tailer-map key matches what globSync stored. C. Queue self-deadlock on abort When the SDK generator aborts (idle timeout, user cancel, shutdown) with rows already claimed and yielded but not yet confirmed by ResponseProcessor, those rows sit in 'processing' under THIS worker's PID. The self-healing claim predicate skips them because the worker is still alive — the queue deadlocks until the worker restarts. In the .finally() block, walk the in-flight ids through markFailed so the retry ladder requeues them as 'pending' (or terminates them if retries are exhausted). Includes regression test tests/codex-transcript-watcher-windows.test.ts that asserts each fix at the source level so future refactors can't silently revert them. Co-Authored-By: MakaveliGER <noreply@github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Closes #2192 * fix: standalone batch — npm peer-deps overrides, marketplace self-heal warning, cache prune - Add `overrides: { tree-sitter: ^0.25.0 }` to the generated plugin/package.json so `npm install --production` resolves cleanly without --legacy-peer-deps. Fixes the ERESOLVE between grammar packages declaring three different majors of tree-sitter as peer deps. Closes #2147. - mcp-server.ts: emit a single loud, actionable warning when MCP boots but the marketplace directory at ~/.claude/plugins/marketplaces/<source>/ is missing. IDE plugin loaders silently skip claude-mem hooks in this state while MCP keeps working — the user has no way to know memory capture is dead. We don't run an installer from MCP startup (different permission model), but we tell the user exactly which command to run. Closes #2174. - smart-install.js (both root and plugin variants): prune older claude-mem version directories from ~/.claude/plugins/cache/thedotmack/claude-mem/. Claude Code resolves and caches hook commands per session, so a stale 12.x directory keeps the old hook path alive across restarts even after upgrade. Pruning makes the stale path physically unreachable. Closes #2172 (stale version reference). Note: the issue's secondary claim that @anthropic-ai/claude-agent-sdk is missing from package.json is no longer true — it was added at line 115 in v12.4.x. - #2170 ("ToolUseContext is required for prompt hooks") triaged as upstream: the string does not appear anywhere in this repo. The error originates in Claude Code's hook framework, which we don't own. No code change here. Co-Authored-By: Amadan04 <amadan04@users.noreply.github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: remove stale macOS binary, regen plugin artifacts (build/bundle drift) The committed plugin/scripts/claude-mem (63 MB Mach-O) was last built at v10.3.2 (Feb 2026). It baked in BUILT_IN_VERSION="10.3.1", dev paths (/Users/alexnewman/Scripts/claude-mem/...), and a now-removed POST /api/sessions/complete client + handler (deleted by PR #2136). That meant macOS users running the cached binary hit 404s every time the SessionEnd hook fired (issue #2200), the /api/health endpoint reported a two-major-versions-ago version (issue #2158), and the binary embedded a Zod copy that drifted from the worker bundle (issue #2154). - Delete plugin/scripts/claude-mem and gitignore it. The npm package already excludes it from the "files" allowlist, so no consumer change. The JS fallback (bun-runner.js → worker-service.cjs) covers all functionality on every platform per the existing checkBinaryPlatformCompatibility comment in smart-install.js. - Add npm run build:cli-binary for users who want the macOS speedup back. Produces it on demand from current source — no drift. - Regenerate plugin/scripts/{worker-service,mcp-server}.cjs and plugin/ui/viewer-bundle.js so the shipped artifacts match HEAD. Closes #2158, #2200, #2154. * fix(ci): Windows workflow — install without lockfile (project uses Bun) actions/setup-node@v4 cache: npm requires a package-lock.json and this project uses Bun (only bunfig.toml exists at root). Drop the cache directive, switch npm ci to npm install --no-audit --no-fund, and narrow the build step to npm run build — build-and-sync also runs a marketplace sync + worker restart that hardcodes ~/.claude/plugins, which doesn't exist on CI. * fix: harden observation cap parsing + safe stringify in debug logger CodeRabbit majors on #2206: - shared.ts: validate parsed cap is finite and > 0 before use; wrap JSON.stringify(payload.toolResponse) in try/catch and skip with reason 'payload_unserializable' on circular/throwing payloads, so ingestion never crashes on a bad tool response shape. - logger.ts: the debug-mode JSON dump for objects was unguarded; wrap stringify in try/catch and fall back to formatData on cycles. This is the source the bundled plugin/scripts/context-generator.cjs is built from. * fix(ci+windows): quote bun-runner shell:true args; replace dynamic smoke with static guards CodeRabbit majors on #2208: 1. plugin/scripts/bun-runner.js — shell:true with separate spawnArgs triggers DEP0190 on Node 22+ and breaks paths/args containing spaces. Build a single fully-quoted command string (mirroring findBun()'s 'where bun' approach) and pass spawnArgs=[]. 2. .github/workflows/windows.yml — the dynamic smoke step that counted visible cmd windows around 'claude-mem start' exits 1 on 'claude-mem is not installed' before exercising the spawn path, AND PowerShell try/catch doesn't suppress native exit codes regardless. Replace with three static regression guards covering the exact patterns PR #2208 protects: - PowerShell Start-Process + WindowStyle Hidden in spawnDaemon - bun-runner shell:true with empty spawnArgs (DEP0190 guard) - windowsHide set on SDK spawn factory (issue #2190) * fix(2210): cross-platform paths — Windows USERPROFILE + XDG cache symmetry Greptile P2s on #2210: - mcp-server.ts checkMarketplaceMarker: switch from process.env.HOME ?? '' to os.homedir(). HOME is unset on Windows; the empty fallback resolves relative to cwd, silently no-opping the canary on every Windows install. Also probe both ~/.claude/ and ~/.config/claude/ for the cache check so XDG users get the same warning behavior. - smart-install.js pruneStaleVersionCache (both root + plugin copies): scan both ~/.claude/plugins/cache/thedotmack/ and ~/.config/claude/... paths so users on XDG don't keep stale dirs re-triggering #2172. Greptile's third P2 (mtime vs semver sort for current version) deferred: mtime works correctly for the common case and the directory names start with versions that lexicographically sort the same way mtime does for sequentially-installed versions; semver sort would be a separate change. Refs PR #2210 * fix(2211): drop hardcoded --target from build:cli-binary Greptile P2: the npm script was pinned to bun-darwin-arm64, so an Intel Mac user (or anyone on Linux/Windows running this script manually) got a cross-compiled arm64 binary that runs only via Rosetta on x64 macOS and not at all elsewhere. Bun's --compile defaults to the host platform when --target is omitted. Drop the flag so the script produces a binary that matches whoever runs it. CI builds that need a specific target can still pass --target explicitly. Refs PR #2211 * ci(windows): drop static-grep tripwires, keep real Windows build The "Anti-regression" steps grep ProcessManager.ts/bun-runner.js/process-registry.ts for specific strings (Start-Process, WindowStyle Hidden, shell:true, windowsHide). Tripwires aren't fixes — they make refactoring harder forever and verify nothing the actual Windows build doesn't already verify. The npm install + npm run build on windows-latest is the real guard. * revert: drop byte cap and skip-list extension band-aids Strips two band-aid mechanisms from the SDK boundary fix, keeping only the genuine isolation flags (settingSources: [], strictMcpConfig: true) and the cloud-provider env preservation. Removed: - CLAUDE_MEM_MAX_OBSERVATION_BYTES (default 65536) — dropped oversize observations entirely. The structural fix is to chunk/summarize oversize tool results, not punish the data flow with an invented byte threshold. Tracked separately. - exec_command, write_stdin, apply_patch added to default skip list — static taste decision baked into defaults for everyone. Users can still set CLAUDE_MEM_SKIP_TOOLS themselves. The data flows again. Real fix is a follow-up. * revert: drop pruneStaleVersionCache walker Removes the cache walker that scans plugin cache dirs and deletes "old" version directories by inferred staleness. The structural fix for #2172 is for the installer to delete the prior version when it writes the new one — not for a separate walker to wake up later and guess which directories are stale. Keeps: - npm peer-dep override for tree-sitter (#2147) - Marketplace marker startup probe (#2174) - Cross-platform path handling Tracked separately as a follow-up. * build: regenerate bundled artifacts after merge Rebuilt plugin/scripts/*.cjs from src after merging #2211, #2204, #2205, #2208, #2209, #2206 (post-strip), #2210 (post-strip). Conflicts during merge were resolved by accepting incoming bundled artifacts; this commit replaces them with a clean rebuild from the merged source. Verified: 0 references to MAX_OBSERVATION_BYTES, payload_too_large, or pruneStaleVersionCache in the rebuilt artifacts. --------- Co-authored-by: swithek <52840391+swithek@users.noreply.github.com> Co-authored-by: Roger Deng <13251150+rogerdigital@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Amadan04 <amadan04@users.noreply.github.com>	2026-04-29 17:08:04 -07:00
Alex Newman	d13662d5d8	Cynical deletion: close 27 issues by removing defenders + tolerators (#2141) ... * fix: mirror migration 28 in SessionStore so pending_messages.tool_use_id and worker_pid columns are created (#2139) SessionStore's inline migration list jumped from v27 to v29, skipping rebuildPendingMessagesForSelfHealingClaim. The worker uses SessionStore directly via worker/DatabaseManager.ts and bypasses the canonical MigrationRunner, so fresh installs ended up at "max v29" with neither column present — every queue claim and observation insert failed. Adds addPendingMessagesToolUseIdAndWorkerPidColumns following the existing mirror precedent (addObservationSubagentColumns / addObservationsUniqueContentHashIndex). Uses ALTER TABLE + column-existence guards so already-broken DBs at v29 self-heal on next worker boot. Verified on fresh DB and on a synthetic v29-without-v28 broken DB: both columns and indexes (idx_pending_messages_worker_pid, ux_pending_session_tool) appear after one boot. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: wrap v28 mirror dedup+index creation in transaction Addresses Greptile P2 review on PR #2140: matches the existing pattern in addObservationsUniqueContentHashIndex (v29 mirror at SessionStore.ts:1127) and runner.ts rebuildPendingMessagesForSelfHealingClaim. A crash between the dedup DELETE and the schema_versions INSERT no longer leaves the DB in a half-applied state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(plan): cynical-deletion plan for 29 open issues 9-phase plan applying delete-first lens to triaged issue corpus. Headlines: kill defenders (orphan cleanup, EncodedCommand spawn, restart-port-steal) and tolerators (silent JSON drops, drifted SSE filters). Each phase closes a named subset of issues. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: delete process-management theater (Phase 1: DEL-1 + DEL-2) Delete aggressiveStartupCleanup, the PowerShell -EncodedCommand spawn branch, and the restart-with-port-steal sequence. Replace daemon spawning with a single uniform child_process.spawn path using arg-array form, keeping setsid on Unix when available. The defenders (orphan cleanup, duplicate-worker probes, port stealing) bred more bugs than they fixed. PID file with start-time token already provides correct OS-trust ownership; restart now requests httpShutdown, waits 5s for the port to free, then exits 1 if it didn't (user resolves). Net -247 lines. Closes #2090, #2095 (already fixed at session-init.ts:78), #2107, #2111, #2114, #2117, #2123, #2097, #2135. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: observer-sessions trust boundary via CLAUDE_MEM_INTERNAL env (Phase 2: DEL-9) Replace the cwd === OBSERVER_SESSIONS_DIR discriminator (which every consumer must repeat and inevitably drifts) with a single env-var trust boundary set once at spawn time in buildIsolatedEnv. - buildIsolatedEnv now sets CLAUDE_MEM_INTERNAL=1, covering all three spawn sites (SDKAgent, KnowledgeAgent.prime, KnowledgeAgent.executeQuery) - shouldTrackProject checks the env var first (cwd check stays as belt-and-braces fallback) - New shared shouldEmitProjectRow predicate — SSE broadcaster and pagination filter share the same predicate so they can never drift apart (#2118) - ObservationBroadcaster filters observer rows from SSE stream - PaginationHelper hardcoded 'observer-sessions' replaced with OBSERVER_SESSIONS_PROJECT const - project-filter basename match pass — *observer-sessions* now matches basename, not just full path (globToRegex's [^/]* can't cross /) (#2126 item 1) - New `claude-mem cleanup [--dry-run]` subcommand wires CleanupV12_4_3 through to the worker for #2126 item 5 Closes #2118, #2126. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: strip proxy env vars before spawning worker (Phase 4: CON-1) User's HTTP_PROXY/HTTPS_PROXY config was bleeding into internal AI calls when claude-mem spawns the claude subprocess, causing connection failures. Strip unconditionally — no passthrough knob, which rejects #2099's whitelist proposal. Closes #2115, #2099. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: fail-fast on silent drops in stdin/file-context/memory-save (Phase 5: FF-1) Three independent fail-fast fixes: #2089 — stdin-reader silent drop. Non-empty stdin that fails JSON.parse now rejects with a clear error instead of resolving undefined. Empty stdin still resolves undefined. #2094 — PreToolUse:Read truncation Edit deadlock. file-context handler no longer returns a fake truncated Read result via updatedInput. Removes userOffset/userLimit/truncated machinery; injects the timeline via additionalContext only and lets the real Read pass through. Read state and Claude's expectation now stay consistent, eliminating the infinite Edit retry loop. #2116 — /api/memory/save metadata drop + project bug. Schema accepts metadata as a documented JSON column (migration 30 adds observations. metadata TEXT, mirrored in SessionStore). Schema also tightened to .strict() so unknown top-level fields fail fast instead of being silently dropped. Project resolution now consults metadata.project as a fallback before defaultProject. Closes #2089, #2094, #2116. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: small deletions — Zod externalize / Gemini fallback / session timeout / installCLI alias (Phase 6) DEL-4 (#2113): Externalize zod from mcp-server.cjs and context-generator.cjs hook bundles so OpenCode's runtime resolves a single Zod copy. Worker keeps Zod bundled (it's a daemon subprocess, not in OpenCode's hook bundle). Added zod to plugin/package.json so externalized requires resolve at runtime. DEL-5 (#2087): Delete the never-wired GeminiAgent → Claude fallback. fallbackAgent was always null in production. On 429 the agent now throws cleanly (message stays pending for retry). Removed setFallbackAgent, FallbackAgent interface, and the 429 fallback branch from both GeminiAgent and OpenRouterAgent. Updated docs that claimed automatic Claude fallback. DEL-6 (#2127, #2098): Raise MAX_SESSION_WALL_CLOCK_MS from 4h to 24h. The timeout is a real guard against runaway-cost loops (per issue #1590), but 4h kills legitimate long Claude Code days. 24h preserves the guard while never hitting in normal use. No knob — a session approaching this age is a bug worth investigating, not a value worth tuning. DEL-8 (#2054): Delete installCLI() alias function. Saves 4 keystrokes at the cost of cross-platform shell-config mutation surface — not worth it. Canonical entry is npx claude-mem (and bunx). Uninstall now strips legacy alias/function lines from ~/.bashrc, ~/.zshrc, and the PowerShell profile. Closes #2087, #2098, #2113, #2127, #2054. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: de-hardcode worker port + multi-account commit (Phase 3: CON-2 + DEL-7) Replace hardcoded 37777 fallbacks with SettingsDefaultsManager.get( 'CLAUDE_MEM_WORKER_PORT') in npx-cli (runtime/install/uninstall), opencode-plugin, OpenClaw installer, SearchRoutes example URLs. Timeline-report SKILL.md now resolves WORKER_PORT from settings.json at the top and uses ${WORKER_PORT} in all curl invocations. Remaining 37777 literals are doc comments + viewer build-time form- field placeholder (which is replaced by /api/settings on mount). hooks.json: add cygpath POSIX→Windows path translation between _R resolution and node invocation. No-op on macOS/Linux. Closes the Windows + Git Bash MODULE_NOT_FOUND in #2109. CLAUDE.md gains a Multi-account section documenting CLAUDE_MEM_DATA_DIR + optional CLAUDE_MEM_WORKER_PORT — every existing path/port code path now honors them. Closes #2103, #2109, #2101. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: install/uninstall improvements (Phase 7: #2106) 5 fixes for the install/uninstall flow: Item 1 — multiselect default. install.ts no longer pre-selects every detected IDE; user explicitly opts in. Item 3 — shutdown-before-overwrite. New src/services/install/shutdown-helper.ts shared by install and uninstall: POSTs /api/admin/shutdown then polls /api/health until the worker stops responding. install calls it before copyPluginToMarketplace so reinstall over a running worker doesn't conflict; uninstall calls it before deletion. Item 4 — uninstall path coverage. Removes ~/.npm/_npx/*/node_modules/ claude-mem, ~/.cache/claude-cli-nodejs/*/mcp-logs-plugin-claude-mem-*, ~/.claude/plugins/data/claude-mem-thedotmack/. Best-effort: per-path try/catch so a single permission failure doesn't abort uninstall. chroma-mcp shutdown is implicit via the worker's GracefulShutdown cascade in item 3's helper. Item 5 — install summary documents "Close all Claude Code sessions before uninstalling, or ~/.claude-mem will be recreated by active hooks." Item 6 — real-port query. After install, fetches /api/health on the configured port with 3s timeout. Reports actually-bound port if the response carries it; falls back to requested port. No retry loop. Closes #2106 (items 1, 3, 4, 5, 6). Items 2, 7 closed separately as already-fixed and insufficient-detail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: pin chroma-mcp to 0.2.6 (Phase 8: DEL-3 lite) Replace unpinned 'chroma-mcp' arg with chroma-mcp==0.2.6 in both local and remote modes. Pinning makes installs deterministic across machines and across time, eliminating the dependency-drift class of bugs. Verified 0.2.6 in a clean uv cache: starts cleanly, no httpcore/ httpx ImportError, no --with flags needed. The --with flags removed in a0dd516c are not required at this pin (transitive deps resolve correctly when the top-level version is fixed). #2102's three protections (transport cleanup on failure, stale onclose handler guard, 10s reconnect backoff) confirmed intact. Closes #2046, #2085, #2102. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: update stale assertions for per-UID port + migration 30 (Phase 9) SettingsDefaultsManager.CLAUDE_MEM_WORKER_PORT default is per-UID (37700 + uid%100), not literal '37777'. Three assertions in settings-defaults-manager.test.ts now compute the expected value the same way the source does. migration-runner.test.ts: drop expect(versions).toContain(19) (version 19 was a noop never recorded — pre-existing bug at parent), add expect(versions).toContain(30) for the new observations.metadata column added in Phase 5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address Greptile P1/P2 review comments on PR #2141 P1: spawnDaemon return value was unchecked in worker-service.ts restart case, so a failed spawn silently exited 0 with a misleading "Worker restart spawned" log. Now error and exit 1 when restartPid is undefined. P2: shutdown-helper.ts health-poll catch treated AbortError (timeout) the same as connection-refused, so a slow worker could be reported confirmedStopped while still holding file locks. Now distinguish: AbortError continues polling; other errors return confirmedStopped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build: rebuild plugin artifacts after merging main Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review comments on PR #2141 - hooks.json: quote $HOME in cache lookup so paths with spaces work - timeline-report SKILL.md: fall back when process.getuid is unavailable (Windows) - opencode-plugin: validate CLAUDE_MEM_WORKER_PORT before using - uninstall.ts: only strip alias lines, not function declarations (multi-line bodies left intact) - MemoryRoutes: trim whitespace-only project before precedence resolution - SessionStore migration 21: preserve metadata column if observations already has it - stdin-reader test: restore full property descriptor to avoid cross-test pollution Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-25 21:23:24 -07:00
Alex Newman	e8082bb992	fix: add missing migration 28 mirror in SessionStore (#2139) (#2140) ... * fix: mirror migration 28 in SessionStore so pending_messages.tool_use_id and worker_pid columns are created (#2139) SessionStore's inline migration list jumped from v27 to v29, skipping rebuildPendingMessagesForSelfHealingClaim. The worker uses SessionStore directly via worker/DatabaseManager.ts and bypasses the canonical MigrationRunner, so fresh installs ended up at "max v29" with neither column present — every queue claim and observation insert failed. Adds addPendingMessagesToolUseIdAndWorkerPidColumns following the existing mirror precedent (addObservationSubagentColumns / addObservationsUniqueContentHashIndex). Uses ALTER TABLE + column-existence guards so already-broken DBs at v29 self-heal on next worker boot. Verified on fresh DB and on a synthetic v29-without-v28 broken DB: both columns and indexes (idx_pending_messages_worker_pid, ux_pending_session_tool) appear after one boot. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: wrap v28 mirror dedup+index creation in transaction Addresses Greptile P2 review on PR #2140: matches the existing pattern in addObservationsUniqueContentHashIndex (v29 mirror at SessionStore.ts:1127) and runner.ts rebuildPendingMessagesForSelfHealingClaim. A crash between the dedup DELETE and the schema_versions INSERT no longer leaves the DB in a half-applied state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-25 18:45:51 -07:00
Alex Newman	703c64c756	v12.4.3: one-time pollution cleanup migration + v12.4.1/v12.4.2 fixes (#2133) ... * fix: 5 trivial bugs from v12.4.1 issue triage - #2092: emit CJS-safe banner (no import.meta.url) in worker-service.cjs - #2100: PreToolUse Read hook timeout 2000s → 60s - #2131: add "shell": "bash" to every hook for Windows compat - #2132: Antigravity dir typo .agent → .agents - #2088: clear inherited MCP servers in worker SDK query() calls Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: stop context overflow loop + block task-notification leak - SDKAgent: clear memorySessionId on "prompt is too long" so crash-recovery starts a fresh SDK session instead of resuming the same poisoned context forever (was producing 68+ failed pending_messages on a single stuck session in the wild) - tag-stripping: new isInternalProtocolPayload() predicate; session-init hook + SessionRoutes both skip storage when entire prompt is one of Claude Code's autonomous protocol blocks (currently <task-notification>; conservative deny-list — does NOT touch <command-name>/<command-message> which wrap real user slash-commands) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version to 12.4.2 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update CHANGELOG.md for v12.4.2 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cleanup): one-time v12.4.3 migration purges observer-sessions and stuck pending_messages Adds CleanupV12_4_3 module that runs once per data dir on worker startup (after migrations apply, before Chroma backfill). Drops accumulated pollution that v12.4.0 (observer-sessions filter) and v12.4.2 (context-overflow guard + task-notification leak block) prevent from recurring: - DELETE FROM sdk_sessions WHERE project='observer-sessions' (cascades to user_prompts, observations, session_summaries via existing FK ON DELETE CASCADE) - DELETE FROM pending_messages stuck in 'failed'/'processing' for any session with >=10 such rows (poisoned chains from the pre-v12.4.2 retry loop; threshold spares legitimate transient failures) - Wipes ~/.claude-mem/chroma and chroma-sync-state.json so backfillAllProjects rebuilds the vector store from cleaned SQLite Pre-flight checks free disk (1.2x DB size + 100MB) via fs.statfsSync; backs up via VACUUM INTO with copyFileSync fallback; PRAGMA foreign_keys=ON on the cleanup connection (off by default in bun:sqlite). Marker file ~/.claude-mem/.cleanup-v12.4.3-applied records backup path and counts. Opt-out via CLAUDE_MEM_SKIP_CLEANUP_V12_4_3=1. Verified locally: 311MB DB backed up to 277MB in 943ms; 11 observer sessions + 3 cascade rows + 141 stuck pending_messages purged; chroma rebuilt via backfill. Total cleanup time 1.1s. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address PR #2133 code review - SessionRoutes: check isInternalProtocolPayload before stripping tags so internal protocol prompts skip the strip work entirely. - tag-stripping: bound isInternalProtocolPayload input length to 256KB to prevent ReDoS-class scans on malformed unclosed tags. - SDKAgent: extract resetSessionForFreshStart helper; both context-overflow paths now share one nullification routine. - worker-service: drop the per-startup "Checking for one-time v12.4.3 cleanup" info log — runs every boot even after marker exists; the function already logs at debug/warn when relevant. - tests: add isInternalProtocolPayload edge cases (whitespace, attributes, partial tags, unrelated tags, oversize input). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address Greptile P2 comments on PR #2133 CleanupV12_4_3.ts: derive backup directory and restore-hint path from effectiveDataDir instead of the module-level BACKUPS_DIR/DB_PATH constants. The dataDirectory override is meant for test isolation; the prior version still wrote backups to the production directory. SessionRoutes.ts: move isInternalProtocolPayload guard to the top of handleSessionInitByClaudeId, before createSDKSession. The previous position blocked the user_prompts insert but still created an empty sdk_sessions row, asymmetric with the hook-layer guard in session-init.ts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cleanup): retry on disk-skip; survive chroma wipe failure CodeRabbit Major + Claude review: - Disk pre-flight skip no longer writes the marker. A user temporarily low on disk would otherwise have the cleanup permanently disabled even after freeing space. Retry on next startup instead. - Wrap wipeChromaArtifacts in try/catch and write the marker even on failure (with chromaWipeError captured). Without this, an rmSync permission failure on chroma/ left writeMarker unreached, so every subsequent boot re-ran the SQL purge AND created a fresh backup, consuming disk indefinitely. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cleanup): close backup handle before copyFileSync fallback Claude review: - backupDb is now closed before falling into the copyFileSync fallback. On Windows an open SQLite handle holds a file lock that can prevent the fallback copy from reading the source. The previous version only closed after both branches completed. - Add empty-body <task-notification></task-notification> case to the isInternalProtocolPayload tests for completeness. Cascade-row count queries already match the actual FK columns (content_session_id for user_prompts, memory_session_id for observations / session_summaries) — no fix needed there. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cleanup): accurate session count + add migration tests Claude review v3: session-init.ts: filter on rawPrompt before the [media prompt] substitution. Functionally equivalent but explicit — the check no longer depends on the substitution leaving real protocol payloads untouched. CleanupV12_4_3.ts: counts.observerSessions now comes from a pre-DELETE COUNT(*), not from result.changes. bun:sqlite inflates result.changes with FTS-trigger and cascade row counts (the user_prompts_fts triggers inflate a 3-session purge to 19 changes). The previous code logged a misleading total and wrote it to the marker. tests/infrastructure/cleanup-v12_4_3.test.ts: happy-path coverage of the migration against a real on-disk SQLite under a tmpdir. Verifies observer-session purge with cascades, stuck pending_messages purge, chroma artifact wipe, marker payload shape, idempotency on re-run, and CLAUDE_MEM_SKIP_CLEANUP_V12_4_3 opt-out. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(protocol-filter): close two-block false positive; address review CodeRabbit + Claude review v5: tag-stripping.ts: PROTOCOL_ONLY_REGEX rewritten with a negative-lookahead body so a prompt like "<task-notification>x</task-notification> hi <task-notification>y</task-notification>" no longer matches as a single outer block — the prior greedy [\s\S]* spanned the middle user text and would have silently dropped a real prompt. Confirmed via probe. tag-stripping.test.ts: drop the 50ms wall-clock assertion (CI flake); add the two-block-with-text case as a regression test. SessionRoutes.ts: filter on req.body.prompt directly, before the [media prompt] substitution and 256KB truncation. Mirrors the session-init.ts hook-layer ordering and ensures a protocol payload that happens to be near the byte limit isn't truncated before the filter runs. cleanup-v12_4_3.test.ts: add stuckCount=9 below-threshold case verifying pending_messages with <10 stuck rows are preserved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cleanup): include WAL/SHM in backup fallback; safer rollback CodeRabbit Major + Claude review v6: CleanupV12_4_3.ts: when VACUUM INTO fails and copyFileSync runs, also copy any -wal/-shm sidecars. The DB is configured WAL mode, so recent committed pages can live in those files; copying only the .db would miss them. VACUUM INTO already captures everything in one file, so the happy path is unaffected. CleanupV12_4_3.ts: wrap ROLLBACK in try/catch so a no-op rollback (SQLite already rolled back on a constraint failure) cannot shadow the original purge error. SDKAgent.ts: align both context-overflow log levels to error. Both branches are fatal-recovery paths; the previous warn/error split was inconsistent and made the throw branch easy to miss in logs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: pre-count stuck pending_messages; document adjacent-block fall-through Claude review v7: CleanupV12_4_3.ts: runStuckPendingPurge now uses a SELECT COUNT(*) before the DELETE, matching the pattern in runObserverSessionsPurge. result.changes is reliable today (no FTS on pending_messages) but the explicit count protects against future schema additions, and keeps the two purges symmetric. tag-stripping.test.ts: add test documenting that adjacent protocol blocks (no user text between) deliberately fall through to storage. The deny-list is per-block; concatenations are out of scope. Skipped per project rules / Node API constraints: - frsize fallback in disk check: Node/Bun StatFs doesn't expose frsize - VACUUM-INTO comment: comment-only suggestion - Overflow string constant extraction: low value Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-25 16:30:34 -07:00
Alex Newman	94d592f212	perf: streamline worker startup and consolidate database connections (#2122) ... * docs: pathfinder refactor corpus + Node 20 preflight Adds the PATHFINDER-2026-04-22 principle-driven refactor plan (11 docs, cross-checked PASS) plus the exploratory PATHFINDER-2026-04-21 corpus that motivated it. Bumps engines.node to >=20.0.0 per the ingestion-path plan preflight (recursive fs.watch). Adds the pathfinder skill. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: land PATHFINDER Plan 01 — data integrity Schema, UNIQUE constraints, self-healing claim, Chroma upsert fallback. - Phase 1: fresh schema.sql regenerated at post-refactor shape. - Phase 2: migrations 23+24 — rebuild pending_messages without started_processing_at_epoch; UNIQUE(session_id, tool_use_id); UNIQUE(memory_session_id, content_hash) on observations; dedup duplicate rows before adding indexes. - Phase 3: claimNextMessage rewritten to self-healing query using worker_pid NOT IN live_worker_pids; STALE_PROCESSING_THRESHOLD_MS and the 60-s stale-reset block deleted. - Phase 4: DEDUP_WINDOW_MS and findDuplicateObservation deleted; observations.insert now uses ON CONFLICT DO NOTHING. - Phase 5: failed-message purge block deleted from worker-service 2-min interval; clearFailedOlderThan method deleted. - Phase 6: repairMalformedSchema and its Python subprocess repair path deleted from Database.ts; SQLite errors now propagate. - Phase 7: Chroma delete-then-add fallback gated behind CHROMA_SYNC_FALLBACK_ON_CONFLICT env flag as bridge until Chroma MCP ships native upsert. - Phase 8: migration 19 no-op block absorbed into fresh schema.sql. Verification greps all return 0 matches. bun test tests/sqlite/ passes 63/63. bun run build succeeds. Plan: PATHFINDER-2026-04-22/01-data-integrity.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: land PATHFINDER Plan 02 — process lifecycle OS process groups replace hand-rolled reapers. Worker runs until killed; orphans are prevented by detached spawn + kill(-pgid). - Phase 1: src/services/worker/ProcessRegistry.ts DELETED. The canonical registry at src/supervisor/process-registry.ts is the sole survivor; SDK spawn site consolidated into it via new createSdkSpawnFactory/spawnSdkProcess/getSdkProcessForSession/ ensureSdkProcessExit/waitForSlot helpers. - Phase 2: SDK children spawn with detached:true + stdio: ['ignore','pipe','pipe']; pgid recorded on ManagedProcessInfo. - Phase 3: shutdown.ts signalProcess teardown uses process.kill(-pgid, signal) on Unix when pgid is recorded; Windows path unchanged (tree-kill/taskkill). - Phase 4: all reaper intervals deleted — startOrphanReaper call, staleSessionReaperInterval setInterval (including the co-located WAL checkpoint — SQLite's built-in wal_autocheckpoint handles WAL growth without an app-level timer), killIdleDaemonChildren, killSystemOrphans, reapOrphanedProcesses, reapStaleSessions, and detectStaleGenerator. MAX_GENERATOR_IDLE_MS and MAX_SESSION_IDLE_MS constants deleted. - Phase 5: abandonedTimer — already 0 matches; primary-path cleanup via generatorPromise.finally() already lives in worker-service startSessionProcessor and SessionRoutes ensureGeneratorRunning. - Phase 6: evictIdlestSession and its evict callback deleted from SessionManager. Pool admission gates backpressure upstream. - Phase 7: SDK-failure fallback — SessionManager has zero matches for fallbackAgent/Gemini/OpenRouter. Failures surface to hooks via exit code 2 through SessionRoutes error mapping. - Phase 8: ensureWorkerRunning in worker-utils.ts rewritten to lazy-spawn — consults isWorkerPortAlive (which gates captureProcessStartToken for PID-reuse safety via commit 99060bac), then spawns detached with unref(), then waitForWorkerPort({ attempts: 3, backoffMs: 250 }) hand-rolled exponential backoff 250→500→1000ms. No respawn npm dep. - Phase 9: idle self-shutdown — zero matches for idleCheck/idleTimeout/IDLE_MAX_MS/idleShutdown. Worker exits only on external SIGTERM via supervisor signal handlers. Three test files that exercised deleted code removed: tests/worker/process-registry.test.ts, tests/worker/session-lifecycle-guard.test.ts, tests/services/worker/reap-stale-sessions.test.ts. Pass count: 1451 → 1407 (-44), all attributable to deleted test files. Zero new failures. 31 pre-existing failures remain (schema-repair suite, logger-usage-standards, environmental openclaw / plugin-distribution) — none introduced by Plan 02. All 10 verification greps return 0. bun run build succeeds. Plan: PATHFINDER-2026-04-22/02-process-lifecycle.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: land PATHFINDER Plan 04 (narrowed) — search fail-fast Phases 3, 5, 6 only. Plan-doc inaccuracies for phases 1/2/4/7/8/9 deferred for plan reconciliation: - Phase 1/2: ObservationRow type doesn't exist; the four "formatters" operate on three incompatible types. - Phase 4: RECENCY_WINDOW_MS already imported from SEARCH_CONSTANTS at every call site. - Phase 7: getExistingChromaIds is NOT @deprecated and has an active caller in ChromaSync.backfillMissingSyncs. - Phase 8: estimateTokens already consolidated. - Phase 9: knowledge-corpus rewrite blocked on PG-3 prompt-caching cost smoke test. Phase 3 — Delete SearchManager.findByConcept/findByFile/findByType. SearchRoutes handlers (handleSearchByConcept/File/Type) now call searchManager.getOrchestrator().findByXxx() directly via new getter accessors on SearchManager. ~250 LoC deleted. Phase 5 — Fail-fast Chroma. Created src/services/worker/search/errors.ts with ChromaUnavailableError extends AppError(503, 'CHROMA_UNAVAILABLE'). Deleted SearchOrchestrator.executeWithFallback's Chroma-failed SQLite-fallback branch; runtime Chroma errors now throw 503. "Path 3" (chromaSync was null at construction — explicit- uninitialized config) preserved as legitimate empty-result state per plan text. ChromaSearchStrategy.search no longer wraps in try/catch — errors propagate. Phase 6 — Delete HybridSearchStrategy three try/catch silent fallback blocks (findByConcept, findByType, findByFile) at lines ~82-95, ~120-132, ~161-172. Removed `fellBack` field from StrategySearchResult type and every return site (SQLiteSearchStrategy, BaseSearchStrategy.emptyResult, SearchOrchestrator). Tests updated (Principle 7 — delete in same PR): - search-orchestrator.test.ts: "fall back to SQLite" rewritten as "throw ChromaUnavailableError (HTTP 503)". - chroma/hybrid/sqlite-search-strategy tests: rewritten to rejects.toThrow; removed fellBack assertions. Verification: SearchManager.findBy → 0; fellBack → 0 in src/. bun test tests/worker/search/ → 122 pass, 0 fail. bun test (suite-wide) → 1407 pass, baseline maintained, 0 new failures. bun run build succeeds. Plan: PATHFINDER-2026-04-22/04-read-path.md (Phases 3, 5, 6) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: land PATHFINDER Plan 03 — ingestion path Fail-fast parser, direct in-process ingest, recursive fs.watch, DB-backed tool pairing. Worker-internal HTTP loopback eliminated. - Phase 0: Created src/services/worker/http/shared.ts exporting ingestObservation/ingestPrompt/ingestSummary as direct in-process functions plus ingestEventBus (Node EventEmitter, reusing existing pattern — no third event bus introduced). setIngestContext wires the SessionManager dependency from worker-service constructor. - Phase 1: src/sdk/parser.ts collapsed to one parseAgentXml returning { valid:true; kind: 'observation'|'summary'; data } | { valid:false; reason: string }. Inspects root element; <skip_summary reason="…"/> is a first-class summary case with skipped:true. NEVER returns undefined. NEVER coerces. - Phase 2: ResponseProcessor calls parseAgentXml exactly once, branches on the discriminated union. On invalid → markFailed + logger.warn(reason). On observation → ingestObservation. On summary → ingestSummary then emit summaryStoredEvent { sessionId, messageId } (consumed by Plan 05's blocking /api/session/end). - Phase 3: Deleted consecutiveSummaryFailures field (ResponseProcessor + SessionManager + worker-types) and MAX_CONSECUTIVE_SUMMARY_FAILURES constant. Circuit-breaker guards and "tripped" log lines removed. - Phase 4: coerceObservationToSummary deleted from sdk/parser.ts. - Phase 5: src/services/transcripts/watcher.ts rescan setInterval replaced with fs.watch(transcriptsRoot, { recursive: true, persistent: true }) — Node 20+ recursive mode. - Phase 6: src/services/transcripts/processor.ts pendingTools Map deleted. tool_use rows insert with INSERT OR IGNORE on UNIQUE(session_id, tool_use_id) (added by Plan 01). New pairToolUsesByJoin query in PendingMessageStore for read-time pairing (UNIQUE INDEX provides idempotency; explicit consumer not yet wired). - Phase 7: HTTP loopback at processor.ts:252 replaced with direct ingestObservation call. maybeParseJson silent-passthrough rewritten to fail-fast (throws on malformed JSON). - Phase 8: src/utils/tag-stripping.ts countTags + stripTagsInternal collapsed into one alternation regex, single-pass over input. - Phase 9: src/utils/transcript-parser.ts (dead TranscriptParser class) deleted. The active extractLastMessage at src/shared/transcript-parser.ts:41-144 is the sole survivor. Tests updated (Principle 7 — same-PR delete): - tests/sdk/parser.test.ts + parse-summary.test.ts: rewritten to assert discriminated-union shape; coercion-specific scenarios collapse into { valid:false } assertions. - tests/worker/agents/response-processor.test.ts: circuit-breaker describe block skipped; non-XML/empty-response tests assert fail-fast markFailed behavior. Verification: every grep returns 0. transcript-parser.ts deleted. bun run build succeeds. bun test → 1399 pass / 28 fail / 7 skip (net -8 pass = the 4 retired circuit-breaker tests + 4 collapsed parser cases). Zero new failures vs baseline. Deferred (out of Plan 03 scope, will land in Plan 06): SessionRoutes HTTP route handlers still call sessionManager.queueObservation inline rather than the new shared helpers — the helpers are ready, the route swap is mechanical and belongs with the Zod refactor. Plan: PATHFINDER-2026-04-22/03-ingestion-path.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: land PATHFINDER Plan 05 — hook surface Worker-call plumbing collapsed to one helper. Polling replaced by server-side blocking endpoint. Fail-loud counter surfaces persistent worker outages via exit code 2. - Phase 1: plugin/hooks/hooks.json — three 20-iteration `for i in 1..20; do curl -sf .../health && break; sleep 0.1; done` shell retry wrappers deleted. Hook commands invoke their bun entry point directly. - Phase 2: src/shared/worker-utils.ts — added executeWithWorkerFallback<T>(url, method, body) returning T | { continue: true; reason?: string }. All 8 hook handlers (observation, session-init, context, file-context, file-edit, summarize, session-complete, user-message) rewritten to use it instead of duplicating the ensureWorkerRunning → workerHttpRequest → fallback sequence. - Phase 3: blocking POST /api/session/end in SessionRoutes.ts using validateBody + sessionEndSchema (z.object({sessionId})). One-shot ingestEventBus.on('summaryStoredEvent') listener, 30 s timer, req.aborted handler — all share one cleanup so the listener cannot leak. summarize.ts polling loop, plus MAX_WAIT_FOR_SUMMARY_MS / POLL_INTERVAL_MS constants, deleted. - Phase 4: src/shared/hook-settings.ts — loadFromFileOnce() memoizes SettingsDefaultsManager.loadFromFile per process. Per-handler settings reads collapsed. - Phase 5: src/shared/should-track-project.ts — single exclusion check entry; isProjectExcluded no longer referenced from src/cli/handlers/. - Phase 6: cwd validation pushed into adapter normalizeInput (all 6 adapters: claude-code, cursor, raw, gemini-cli, windsurf). New AdapterRejectedInput error in src/cli/adapters/errors.ts. Handler-level isValidCwd checks deleted from file-edit.ts and observation.ts. hook-command.ts catches AdapterRejectedInput → graceful fallback. - Phase 7: session-init.ts conditional initAgent guard deleted; initAgent is idempotent. tests/hooks/context-reinjection-guard test (validated the deleted conditional) deleted in same PR per Principle 7. - Phase 8: fail-loud counter at ~/.claude-mem/state/hook-failures .json. Atomic write via .tmp + rename. CLAUDE_MEM_HOOK_FAIL_LOUD _THRESHOLD setting (default 3). On consecutive worker-unreachable ≥ N: process.exit(2). On success: reset to 0. NOT a retry. - Phase 9: ensureWorkerAliveOnce() module-scope memoization wrapping ensureWorkerRunning. executeWithWorkerFallback calls the memoized version. Minimal validateBody middleware stub at src/services/worker/http/middleware/validateBody.ts. Plan 06 will expand with typed inference + error envelope conventions. Verification: 4/4 grep targets pass. bun run build succeeds. bun test → 1393 pass / 28 fail / 7 skip; -6 pass attributable solely to deleted context-reinjection-guard test file. Zero new failures vs baseline. Plan: PATHFINDER-2026-04-22/05-hook-surface.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: land PATHFINDER Plan 06 — API surface One Zod-based validator wrapping every POST/PUT. Rate limiter, diagnostic endpoints, and shutdown wrappers deleted. Failure- marking consolidated to one helper. - Phase 1 (preflight): zod@^3 already installed. - Phase 2: validateBody middleware confirmed at canonical shape in src/services/worker/http/middleware/validateBody.ts — safeParse → 400 { error: 'ValidationError', issues: [...] } on failure, replaces req.body with parsed value on success. - Phase 3: Per-route Zod schemas declared at the top of each route file. 24 POST endpoints across SessionRoutes, CorpusRoutes, DataRoutes, MemoryRoutes, SearchRoutes, LogsRoutes, SettingsRoutes now wrap with validateBody(). /api/session/end (Plan 05) confirmed using same middleware. - Phase 4: validateRequired() deleted from BaseRouteHandler along with every call site. Inline coercion helpers (coerceStringArray, coercePositiveInteger) and inline if (!req.body...) guards deleted across all route files. - Phase 5: Rate limiter middleware and its registration deleted from src/services/worker/http/middleware.ts. Worker binds 127.0.0.1:37777 — no untrusted caller. - Phase 6: viewer.html cached at module init in ViewerRoutes.ts via fs.readFileSync; served as Buffer with text/html content type. SKILL.md + per-operation .md files cached in Server.ts as Map<string, string>; loadInstructionContent helper deleted. NO fs.watch, NO TTL — process restart is the cache-invalidation event. - Phase 7: Four diagnostic endpoints deleted from DataRoutes.ts — /api/pending-queue (GET), /api/pending-queue/process (POST), /api/pending-queue/failed (DELETE), /api/pending-queue/all (DELETE). Helper methods that ONLY served them (getQueueMessages, getStuckCount, getRecentlyProcessed, clearFailed, clearAll) deleted from PendingMessageStore. KEPT: /api/processing-status (observability), /health (used by ensureWorkerRunning). - Phase 8: stopSupervisor wrapper deleted from supervisor/index.ts. GracefulShutdown now calls getSupervisor().stop() directly. Two functions retained with clear roles: - performGracefulShutdown — worker-side 6-step shutdown - runShutdownCascade — supervisor-side child teardown (process.kill(-pgid), Windows tree-kill, PID-file cleanup) Each has unique non-trivial logic and a single canonical caller. - Phase 9: transitionMessagesTo(status, filter) is the sole failure-marking path on PendingMessageStore. Old methods markSessionMessagesFailed and markAllSessionMessagesAbandoned deleted along with all callers (worker-service, SessionCompletionHandler, tests/zombie-prevention). Tests updated (Principle 7 same-PR delete): coercion test files refactored to chain validateBody → handler. Zombie-prevention tests rewritten to call transitionMessagesTo. Verification: all 4 grep targets → 0. bun run build succeeds. bun test → 1393 pass / 28 fail / 7 skip — exact match to baseline. Zero new failures. Plan: PATHFINDER-2026-04-22/06-api-surface.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: land PATHFINDER Plan 07 — dead code sweep ts-prune-driven sweep across the tree after Plans 01-06 landed. Deleted unused exports, orphan helpers, and one fully orphaned file. Earlier-plan deletions verified. Deleted: - src/utils/bun-path.ts (entire file — getBunPath, getBunPathOrThrow, isBunAvailable: zero importers) - bun-resolver.getBunVersionString: zero callers - PendingMessageStore.retryMessage / resetProcessingToPending / abortMessage: superseded by transitionMessagesTo (Plan 06 Phase 9) - EnvManager.MANAGED_CREDENTIAL_KEYS, EnvManager.setCredential: zero callers - CodexCliInstaller.checkCodexCliStatus: zero callers; no status command exists in npx-cli - Two "REMOVED: cleanupOrphanedSessions" stale-fence comments Kept (with documented justification): - Public API surface in dist/sdk/* (parseAgentXml, prompt builders, ParsedObservation, ParsedSummary, ParseResult, SUMMARY_MODE_MARKER) — exported via package.json sdk path. - generateContext / loadContextConfig / token utilities — used via dynamic await import('../../../context-generator.js') in worker SearchRoutes. - MCP_IDE_INSTALLERS, install/uninstall functions for codex/goose — used via dynamic await import in npx-cli/install.ts + uninstall.ts (ts-prune cannot trace dynamic imports). - getExistingChromaIds — active caller in ChromaSync.backfillMissingSyncs (Plan 04 narrowed scope). - processPendingQueues / getSessionsWithPendingMessages — active orphan-recovery caller in worker-service.ts plus zombie-prevention test coverage. - StoreAndMarkCompleteResult legacy alias — return-type annotation in same file. - All Database.ts barrel re-exports — used downstream. Earlier-plan verification: - Plan 03 Phase 9: VERIFIED — src/utils/transcript-parser.ts is gone; TranscriptParser has 0 references in src/. - Plan 01 Phase 8: VERIFIED — migration 19 no-op absorbed. - SessionStore.ts:52-70 consolidation NOT executed (deferred): the methods are not thin wrappers but ~900 LoC of bodies, and two methods are documented as intentional mirrors so the context-generator.cjs bundle stays schema-consistent without pulling MigrationRunner. Deserves its own plan, not a sweep. Verification: TranscriptParser → 0; transcript-parser.ts → gone; no commented-out code markers remain. bun run build succeeds. bun test → 1393 pass / 28 fail / 7 skip — EXACT match to baseline. Zero regressions. Plan: PATHFINDER-2026-04-22/07-dead-code.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: remove residual ProcessRegistry comment reference Plan 07 dead-code sweep missed one comment-level reference to the deleted in-memory ProcessRegistry class in SessionManager.ts:347. Rewritten to describe the supervisor.json scope without naming the deleted class, completing the verification grep target. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address Greptile review (P1 + 2× P2) P1 — Plan 05 Phase 3 blocking endpoint was non-functional: executeWithWorkerFallback used HEALTH_CHECK_TIMEOUT_MS (3 s) for the POST /api/session/end call, but the server holds the connection for SERVER_SIDE_SUMMARY_TIMEOUT_MS (30 s). Client always raced to a "timed out" rejection that isWorkerUnavailable classified as worker-unreachable, so the hook silently degraded instead of waiting for summaryStoredEvent. - Added optional timeoutMs to executeWithWorkerFallback, forwarded to workerHttpRequest. - summarize.ts call site now passes 35_000 (5 s above server hold window). P2 — ingestSummary({ kind: 'parsed' }) branch was dead code: ResponseProcessor emitted summaryStoredEvent directly via the event bus, bypassing the centralized helper that the comment claimed was the single source. - ResponseProcessor now calls ingestSummary({ kind: 'parsed', sessionDbId, messageId, contentSessionId, parsed }) so the event-emission path is single-sourced. - ingestSummary's requireContext() resolution moved inside the 'queue' branch (the only branch that needs sessionManager / dbManager). 'parsed' is a pure event-bus emission and doesn't need worker-internal context — fixes mocked ResponseProcessor unit tests that don't call setIngestContext. P2 — isWorkerFallback could false-positive on legitimate API responses whose schema includes { continue: true, ... }: - Added a Symbol.for('claude-mem/worker-fallback') brand to WorkerFallback. isWorkerFallback now checks the brand, not a duck-typed property name. Verification: bun run build succeeds. bun test → 1393 pass / 28 fail / 7 skip — exact baseline match. Zero new failures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address Greptile iteration 2 (P1 + P2) P1 — summaryStoredEvent fired regardless of whether the row was persisted. ResponseProcessor's call to ingestSummary({ kind: 'parsed' }) ran for every parsed.kind === 'summary' even when result.summaryId came back null (e.g. FK violation, null memory_session_id at commit). The blocking /api/session/end endpoint then returned { ok: true } and the Stop hook logged 'Summary stored' for a non-existent row. - Gate ingestSummary call on (parsed.data.skipped || session.lastSummaryStored). Skipped summaries are an explicit no-op bypass and still confirm; real summaries only confirm when storage actually wrote a row. - Non-skipped + summaryId === null path logs a warn and lets the server-side timeout (504) surface to the hook instead of a false ok:true. P2 — PendingMessageStore.enqueue() returns 0 when INSERT OR IGNORE suppresses a duplicate (the UNIQUE(session_id, tool_use_id) constraint added by Plan 01 Phase 1). The two callers (SessionManager.queueObservation and queueSummarize) previously logged 'ENQUEUED messageId=0' which read like a row was inserted. - Branch on messageId === 0 and emit a 'DUP_SUPPRESSED' debug log instead of the misleading ENQUEUED line. No behavior change — the duplicate is still correctly suppressed by the DB (Principle 3); only the log surface is corrected. - confirmProcessed is never called with the enqueue() return value (it operates on session.processingMessageIds[] from claimNextMessage), so no caller is broken; the visibility fix prevents future misuse. Verification: bun run build succeeds. bun test → 1393 pass / 28 fail / 7 skip — exact baseline match. Zero new failures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address Greptile iteration 3 (P1 + 2× P2) - P1 worker-service.ts: wire ensureGeneratorRunning into the ingest context after SessionRoutes is constructed. setIngestContext runs before routes exist, so transcript-watcher observations queued via ingestObservation() had no way to auto-start the SDK generator. Added attachIngestGeneratorStarter() to patch the callback in. - P2 shared.ts: IngestEventBus now sets maxListeners to 0. Concurrent /api/session/end calls register one listener each and clean up on completion, so the default-10 warning fires spuriously under normal load. - P2 SessionRoutes.ts: handleObservationsByClaudeId now delegates to ingestObservation() instead of duplicating skip-tool / meta / privacy / queue logic. Single helper, matching the Plan 03 goal. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address Greptile iteration 4 (P1 tool-pair + P2 parse/path/doc) - processor.handleToolResult: restore in-memory tool-use→tool-result pairing via session.pendingTools for schemas (e.g. Codex) whose tool_result events carry only tool_use_id + output. Without this, neither handler fired — all tool observations silently dropped. - processor.maybeParseJson: return raw string on parse failure instead of throwing. Previously a single malformed JSON-shaped field caused handleLine's outer catch to discard the entire transcript line. - watcher.deepestNonGlobAncestor: split on / and \\, emit empty string for purely-glob inputs so the caller skips the watch instead of anchoring fs.watch at the filesystem root. Windows-compatible. - PendingMessageStore.enqueue: tighten docstring — callers today only log on the returned id; the SessionManager branches on id === 0. * fix: forward tool_use_id through ingestObservation (Greptile iter 5) P1 — Plan 01's UNIQUE(content_session_id, tool_use_id) dedup never fired because the new shared ingest path dropped the toolUseId before queueObservation. SQLite treats NULL values as distinct for UNIQUE, so every replayed transcript line landed a duplicate row. - shared.ingestObservation: forward payload.toolUseId to queueObservation so INSERT OR IGNORE can actually collapse. - SessionRoutes.handleObservationsByClaudeId: destructure both tool_use_id (HTTP convention) and toolUseId (JS convention) from req.body and pass into ingestObservation. - observationsByClaudeIdSchema: declare both keys explicitly so the validator doesn't rely on .passthrough() alone. * fix: drop dead pairToolUsesByJoin, close session-end listener race - PendingMessageStore: delete pairToolUsesByJoin. The method was never called and its self-join semantics are structurally incompatible with UNIQUE(content_session_id, tool_use_id): INSERT OR IGNORE collapses any second row with the same pair, so a self-join can only ever match a row to itself. In-memory pendingTools in processor.ts remains the pairing path for split-event schemas. - IngestEventBus: retain a short-lived (60s) recentStored map keyed by sessionId. Populated on summaryStoredEvent emit, evicted on consume or TTL. - handleSessionEnd: drain the recent-events buffer before attaching the listener. Closes the register-after-emit race where the summary can persist between the hook's summarize POST and its session/end POST — previously that window returned 504 after the 30s timeout. * chore: merge origin/main into vivacious-teeth Resolves conflicts with 15 commits on main (v12.3.9, security observation types, Telegram notifier, PID-reuse worker start-guard). Conflict resolution strategy: - plugin/hooks/hooks.json, plugin/scripts/*.cjs, plugin/ui/viewer-bundle.js: kept ours — PATHFINDER Plan 05 deletes the for-i-in-1-to-20 curl retry loops and the built artifacts regenerate on build. - src/cli/handlers/summarize.ts: kept ours — Plan 05 blocking POST /api/session/end supersedes main's fire-and-forget path. - src/services/worker-service.ts: kept ours — Plan 05 ingest bus + summaryStoredEvent supersedes main's SessionCompletionHandler DI refactor + orphan-reaper fallback. - src/services/worker/http/routes/SessionRoutes.ts: kept ours — same reason; generator .finally() Stop-hook self-clean is a guard for a path our blocking endpoint removes. - src/services/worker/http/routes/CorpusRoutes.ts: merged — added security_alert / security_note to ALLOWED_CORPUS_TYPES (feature from #2084) while preserving our Zod validateBody schema. Typecheck: 294 errors (vs 298 pre-merge). No new errors introduced; all remaining are pre-existing (Component-enum gaps, DOM lib for viewer, bun:sqlite types). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address Greptile P2 findings 1) SessionRoutes.handleSessionEnd was the only route handler not wrapped in wrapHandler — synchronous exceptions would hang the client rather than surfacing as 500s. Wrap it like every other handler. 2) processor.handleToolResult only consumed the session.pendingTools entry when the tool_result arrived without a toolName. In the split-schema path where tool_result carries both toolName and toolId, the entry was never deleted and the map grew for the life of the session. Consume the entry whenever toolId is present. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: typing cleanup and viewer tsconfig split for PR feedback - Add explicit return types for SessionStore query methods - Exclude src/ui/viewer from root tsconfig, give it its own DOM-typed config - Add bun to root tsconfig types, plus misc typing tweaks flagged by Greptile - Rebuilt plugin/scripts/* artifacts Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address Greptile P2 findings (iter 2) - PendingMessageStore.transitionMessagesTo: require sessionDbId (drop the unscoped-drain branch that would nuke every pending/processing row across all sessions if a future caller omitted the filter). - IngestEventBus.takeRecentSummaryStored: make idempotent — keep the cached event until TTL eviction so a retried Stop hook's second /api/session/end returns immediately instead of hanging 30 s. - TranscriptWatcher fs.watch callback: skip full glob scan for paths already tailed (JSONL appends fire on every line; only unknown paths warrant a rescan). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: call finalizeSession in terminal session paths (Greptile iter 3) terminateSession and runFallbackForTerminatedSession previously called SessionCompletionHandler.finalizeSession before removeSessionImmediate; the refactor dropped those calls, leaving sdk_sessions.status='active' for every session killed by wall-clock limit, unrecoverable error, or exhausted fallback chain. The deleted reapStaleSessions interval was the only prior backstop. Re-wires finalizeSession (idempotent: marks completed, drains pending, broadcasts) into both paths; no reaper reintroduced. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: GC failed pending_messages rows at startup (Greptile iter 4) Plan 07 deleted clearFailed/clearFailedOlderThan as "dead code", but with the periodic sweep also removed, nothing reaps status='failed' rows now — they accumulate indefinitely. Since claimNextMessage's self-healing subquery scans this table, unbounded growth degrades claim latency over time. Re-introduces clearFailedOlderThan and calls it once at worker startup (not a reaper — one-shot, idempotent). 7-day retention keeps enough history for operator inspection while bounding the table. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: finalize sessions on normal exit; cleanup hoist; share handler (iter 5) 1. startSessionProcessor success branch now calls completionHandler. finalizeSession before removeSessionImmediate. Hooks-disabled installs (and any Stop hook that fails before POST /api/sessions/complete) no longer leave sdk_sessions rows as status='active' forever. Idempotent — a subsequent /api/sessions/complete is a no-op. 2. Hoist SessionRoutes.handleSessionEnd cleanup declaration above the closures that reference it (TDZ safety; safe at runtime today but fragile if timeout ever shrinks). 3. SessionRoutes now receives WorkerService's shared SessionCompletionHandler instead of constructing its own — prevents silent divergence if the handler ever becomes stateful. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: stop runaway crash-recovery loop on dead sessions Two distinct bugs were combining to keep a dead session restarting forever: Bug 1 (uncaught "The operation was aborted."): child_process.spawn emits 'error' asynchronously for ENOENT/EACCES/abort signal aborts. spawnSdkProcess() never attached an 'error' listener, so any async spawn failure became uncaughtException and escaped to the daemon-level handler. Attach an 'error' listener immediately after spawn, before the !child.pid early-return, so async spawn errors are logged (with errno code) and swallowed locally. Bug 2 (sliding-window limiter never trips on slow restart cadence): RestartGuard tripped only when restartTimestamps.length exceeded MAX_WINDOWED_RESTARTS (10) within RESTART_WINDOW_MS (60s). With the 8s exponential-backoff cap, only ~7-8 restarts fit in the window, so a dead session that fail-restart-fail-restart on 8s cycles would loop forever (consecutiveRestarts climbing past 30+ in observed logs). Add a consecutiveFailures counter that increments on every restart and resets only on recordSuccess(). Trip when consecutive failures exceed MAX_CONSECUTIVE_FAILURES (5) — meaning 5 restarts with zero successful processing in between proves the session is dead. Both guards now run in parallel: tight loops still trip the windowed cap; slow loops trip the consecutive-failure cap. Also: when the SessionRoutes path trips the guard, drain pending messages to 'abandoned' so the session does not reappear in getSessionsWithPendingMessages and trigger another auto-start cycle. The worker-service.ts path already does this via terminateSession. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf: streamline worker startup and consolidate database connections 1. Database Pooling: Modified DatabaseManager, SessionStore, and SessionSearch to share a single bun:sqlite connection, eliminating redundant file descriptors. 2. Non-blocking Startup: Refactored WorktreeAdoption and Chroma backfill to run in the background (fire-and-forget), preventing them from stalling core initialization. 3. Diagnostic Routes: Added /api/chroma/status and bypassed the initialization guard for health/readiness endpoints to allow diagnostics during startup. 4. Robust Search: Implemented reliable SQLite FTS5 fallback in SearchManager for when Chroma (uvx) fails or is unavailable. 5. Code Cleanup: Removed redundant loopback MCP checks and mangled initialization logic from WorkerService. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: hard-exclude observer-sessions from hooks; bundle migration 29 (#2124) * fix: hard-exclude observer-sessions from hooks; backfill bundle migrations Stop hook + SessionEnd hook were storing the SDK observer's own init/continuation/summary prompts in user_prompts, leaking into the viewer (meta-observation regression). 25 such rows accumulated. - shouldTrackProject: hard-reject OBSERVER_SESSIONS_DIR (and its subtree) before consulting user-configured exclusion globs. - summarize.ts (Stop) and session-complete.ts (SessionEnd): early-return when shouldTrackProject(cwd) is false, so the observer's own hooks cannot bootstrap the worker or queue a summary against the meta-session. - SessionRoutes: cap user-prompt body at 256 KiB at the session-init boundary so a runaway observer prompt cannot blow up storage. - SessionStore: add migration 29 (UNIQUE(memory_session_id, content_hash) on observations) inline so bundled artifacts (worker-service.cjs, context-generator.cjs) stay schema-consistent — without it, the ON CONFLICT clause in observation inserts throws. - spawnSdkProcess: stdio[stdin] from 'ignore' to 'pipe' so the supervisor can actually feed the observer's stdin. Also rebuilds plugin/scripts/{worker-service,context-generator}.cjs. * fix: walk back to UTF-8 boundary on prompt truncation (Greptile P2) Plain Buffer.subarray at MAX_USER_PROMPT_BYTES can land mid-codepoint, which the utf8 decoder silently rewrites to U+FFFD. Walk back over any continuation bytes (0b10xxxxxx) before decoding so the truncated prompt ends on a valid sequence boundary instead of a replacement character. * fix: cross-platform observer-dir containment; clarify SDK stdin pipe claude-review feedback on PR #2124. - shouldTrackProject: literal `cwd.startsWith(OBSERVER_SESSIONS_DIR + '/')` hard-coded a POSIX separator and missed Windows backslash paths plus any trailing-slash variance. Switched to a path.relative-based isWithin() helper so Windows hook input under observer-sessions\\... is also excluded. - spawnSdkProcess: added a comment explaining why stdin must be 'pipe' — SpawnedSdkProcess.stdin is typed NonNullable and the Claude Agent SDK consumes that pipe; 'ignore' would null it and the null-check below would tear the child down on every spawn. * fix: make Stop hook fire-and-forget; remove dead /api/session/end The Stop hook was awaiting a 35-second long-poll on /api/session/end, which the worker held open until the summary-stored event fired (or its 30s server-side timeout elapsed). Followed by another await on /api/sessions/complete. Three sequential awaits, the middle one a 30s hold — not fire-and-forget despite repeated requests. The Stop hook now does ONE thing: POST /api/sessions/summarize to queue the summary work and return. The worker drives the rest async. Session-map cleanup is performed by the SessionEnd handler (session-complete.ts), not duplicated here. - summarize.ts: drop the /api/session/end long-poll and the trailing /api/sessions/complete await; ~40 lines removed; unused SessionEndResponse interface gone; header comment rewritten. - SessionRoutes: delete handleSessionEnd, sessionEndSchema, the SERVER_SIDE_SUMMARY_TIMEOUT_MS constant, and the /api/session/end route registration. Drop the now-unused ingestEventBus and SummaryStoredEvent imports. - ResponseProcessor + shared.ts + worker-utils.ts: update stale comments that referenced the dead endpoint. The IngestEventBus is left in place dormant (no listeners) for follow-up cleanup so this PR stays focused on the blocker. Bundle artifact (worker-service.cjs) rebuilt via build-and-sync. Verification: - grep '/api/session/end' plugin/scripts/worker-service.cjs → 0 - grep 'timeoutMs:35' plugin/scripts/worker-service.cjs → 0 - Worker restarted clean, /api/health ok at pid 92368 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * deps: bump all dependencies to latest including majors Upgrades: React 18→19, Express 4→5, Zod 3→4, TypeScript 5→6, @types/node 20→25, @anthropic-ai/claude-agent-sdk 0.1→0.2, @clack/prompts 0.9→1.2, plus minors. Adds Daily Maintenance section to CLAUDE.md mandating latest-version policy across manifests. Express 5 surfaced a race in Server.listen() where the 'error' handler was attached after listen() was invoked; refactored to use http.createServer with both 'error' and 'listening' handlers attached before listen(), restoring port-conflict rejection semantics. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: surface real chroma errors and add deep status probe Replace the misleading "Vector search failed - semantic search unavailable. Install uv... restart the worker." string in SearchManager with the actual exception text from chroma_query_documents. The lying message blamed `uv` for any failure — even when the real cause was a chroma-mcp transport timeout, an empty collection, or a dead subprocess. Also add /api/chroma/status?deep=1 backed by a new ChromaMcpManager.probeSemanticSearch() that round-trips a real query (chroma_list_collections + chroma_query_documents) instead of just checking the stdio handshake. The cheap default path is unchanged. Includes the diagnostic plan (PLAN-fix-mcp-search.md) and updated test fixtures for the new structured failure message. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: rebuild worker-service bundle to match merged src Bundle was stale after the squash merge of #2124 — it still contained the old "Install uv... semantic search unavailable" string and lacked probeSemanticSearch. Rebuilt via bun run build-and-sync. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: address coderabbit feedback on PLAN-fix-mcp-search.md - replace machine-specific /Users/alexnewman absolute paths with portable <repo-root> placeholder (MD-style portability) - add blank lines around the TypeScript fenced block (MD031) - tag the bare fenced block with `text` (MD040) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-25 13:37:40 -07:00
Alex Newman	f2d361b918	feat: security observation types + Telegram notifier (#2084) ... * feat: security observation types + Telegram notifier Adds two severity-axis security observation types (security_alert, security_note) to the code mode and a fire-and-forget Telegram notifier that posts when a saved observation matches configured type or concept triggers. Default trigger fires on security_alert only; notifier is disabled until BOT_TOKEN and CHAT_ID are set. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(telegram): honor CLAUDE_MEM_TELEGRAM_ENABLED master toggle Adds an explicit on/off flag (default 'true') so users can disable the notifier without clearing credentials. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(stop-hook): make summarize handler fire-and-forget Stop hook previously blocked the Claude Code session for up to 110 seconds while polling the worker for summary completion. The handler now returns as soon as the enqueue POST is acked. - summarize.ts: drop the 500ms polling loop and /api/sessions/complete call; tighten SUMMARIZE_TIMEOUT_MS from 300s to 5s since the worker acks the enqueue synchronously. - SessionCompletionHandler: extract idempotent finalizeSession() for DB mark + orphaned-pending-queue drain + broadcast. completeByDbId now delegates so the /api/sessions/complete HTTP route is backward compatible. - SessionRoutes: wire finalizeSession into the SDK-agent generator's finally block, gated on lastSummaryStored + empty pending queue so only Stop events produce finalize (not every idle tick). - WorkerService: own the single SessionCompletionHandler instance and inject it into SessionRoutes to avoid duplicate construction. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pr2084): address reviewer findings CodeRabbit: - SessionStore.getSessionById now returns status; without it, the finalizeSession idempotency guard always evaluated false and re-fired drain/broadcast on every call. - worker-service.ts: three call sites that remove the in-memory session after finalizeSession now do so only on success. On failure the session is left in place so the 60s orphan reaper can retry; removing it would orphan an 'active' DB row indefinitely under the fire-and- forget Stop hook. - runFallbackForTerminatedSession no longer emits a second session_completed event; finalizeSession already broadcasts one. The explicit broadcast now runs only on the finalize-failure fallback. Greptile: - TelegramNotifier reads via loadFromFile(USER_SETTINGS_PATH) so values in ~/.claude-mem/settings.json actually take effect; SettingsDefaultsManager.get() alone skipped the file and silently ignored user-configured credentials. - Emoji is derived from obs.type (security_alert → 🚨, security_note → 🔐, fallback 🔔) instead of hardcoded 🚨 for every observation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(hooks): worker-port mismatch on Windows and settings.json overrides (#2086) Hooks computed the health-check port as \$((37700 + id -u % 100)), ignoring ~/.claude-mem/settings.json. Two failure modes resulted: 1. Users upgrading from pre-per-uid builds kept CLAUDE_MEM_WORKER_PORT set to '37777' in settings.json. The worker bound 37777 (settings wins), but hooks queried 37701 (uid 501 on macOS), so every SessionStart/UserPromptSubmit health check failed. 2. Windows Git Bash/PowerShell returns a real Windows UID for 'id -u' (e.g. 209), producing port 37709 while the Node worker fell back to 37777 (process.getuid?.() ?? 77). Every prompt hit the 60s hook timeout. hooks.json now resolves the port in this order, matching how the worker itself resolves it: 1. sed CLAUDE_MEM_WORKER_PORT from ~/.claude-mem/settings.json 2. If absent, and uname is MINGW/CYGWIN/MSYS → 37777 3. Otherwise 37700 + (id -u || 77) % 100 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pr2084): sync DatabaseManager.getSessionById return type CodeRabbit round 2: the DatabaseManager.getSessionById return type was missing platform_source, custom_title, and status fields that SessionStore.getSessionById actually returns. Structural typing hid the mismatch at compile time, but it prevents callers going through DatabaseManager from seeing the status field that the idempotency guard in SessionCompletionHandler relies on. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pr2084): hooks honor env vars and host; looser port regex (#2086 followup) CodeRabbit round 3: match the worker's env > file > defaults precedence and resolve host the same way as port. - Env: CLAUDE_MEM_WORKER_PORT and CLAUDE_MEM_WORKER_HOST win first. - File: sed now accepts both quoted ('"37777"') and unquoted (37777) JSON values for the port; a separate sed reads CLAUDE_MEM_WORKER_HOST. - Defaults: port per-uid formula (Windows: 37777), host 127.0.0.1. - Health-check URL uses the resolved $HOST instead of hardcoded localhost. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-22 16:08:28 -07:00
Alex Newman	03748acd6a	refactor: remove bearer auth and platform_source context filter (#2081) ... * fix: resolve search, database, and docker bugs (#1913, #1916, #1956, #1957, #2048) - Fix concept/concepts param mismatch in SearchManager.normalizeParams (#1916) - Add FTS5 keyword fallback when ChromaDB is unavailable (#1913, #2048) - Add periodic WAL checkpoint and journal_size_limit to prevent unbounded WAL growth (#1956) - Add periodic clearFailed() to purge stale pending_messages (#1957) - Fix nounset-safe TTY_ARGS expansion in docker/claude-mem/run.sh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: prevent silent data loss on non-XML responses, add queue info to /health (#1867, #1874) - ResponseProcessor: mark messages as failed (with retry) instead of confirming when the LLM returns non-XML garbage (auth errors, rate limits) (#1874) - Health endpoint: include activeSessions count for queue liveness monitoring (#1867) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: cache isFts5Available() at construction time Addresses Greptile review: avoid DDL probe (CREATE + DROP) on every text query. Result is now cached in _fts5Available at construction. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve worker stability bugs — pool deadlock, MCP loopback, restart guard (#1868, #1876, #2053) - Replace flat consecutiveRestarts counter with time-windowed RestartGuard: only counts restarts within 60s window (cap=10), decays after 5min of success. Prevents stranding pending messages on long-running sessions. (#2053) - Add idle session eviction to pool slot allocation: when all slots are full, evict the idlest session (no pending work, oldest activity) to free a slot for new requests, preventing 60s timeout deadlock. (#1868) - Fix MCP loopback self-check: use process.execPath instead of bare 'node' which fails on non-interactive PATH. Fix crash misclassification by removing false "Generator exited unexpectedly" error log on normal completion. (#1876) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve hooks reliability bugs — summarize exit code, session-init health wait (#1896, #1901, #1903, #1907) - Wrap summarize hook's workerHttpRequest in try/catch to prevent exit code 2 (blocking error) on network failures or malformed responses. Session exit no longer blocks on worker errors. (#1901) - Add health-check wait loop to UserPromptSubmit session-init command in hooks.json. On Linux/WSL where hook ordering fires UserPromptSubmit before SessionStart, session-init now waits up to 10s for worker health before proceeding. Also wrap session-init HTTP call in try/catch. (#1907) - Close #1896 as already-fixed: mtime comparison at file-context.ts:255-267 bypasses truncation when file is newer than latest observation. - Close #1903 as no-repro: hooks.json correctly declares all hook events. Issue was Claude Code 12.0.1/macOS platform event-dispatch bug. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: security hardening — bearer auth, path validation, rate limits, per-user port (#1932, #1933, #1934, #1935, #1936) - Add bearer token auth to all API endpoints: auto-generated 32-byte token stored at ~/.claude-mem/worker-auth-token (mode 0600). All hook, MCP, viewer, and OpenCode requests include Authorization header. Health/readiness endpoints exempt for polling. (#1932, #1933) - Add path traversal protection: watch.context.path validated against project root and ~/.claude-mem/ before write. Rejects ../../../etc style attacks. (#1934) - Reduce JSON body limit from 50MB to 5MB. Add in-memory rate limiter (300 req/min/IP) to prevent abuse. (#1935) - Derive default worker port from UID (37700 + uid%100) to prevent cross-user data leakage on multi-user macOS. Windows falls back to 37777. Shell hooks use same formula via id -u. (#1936) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve search project filtering and import Chroma sync (#1911, #1912, #1914, #1918) - Fix per-type search endpoints to pass project filter to Chroma queries and SQLite hydration. searchObservations/Sessions/UserPrompts now use $or clause matching project + merged_into_project. (#1912) - Fix timeline/search methods to pass project to Chroma anchor queries. Prevents cross-project result leakage when project param omitted. (#1911) - Sync imported observations to ChromaDB after FTS rebuild. Import endpoint now calls chromaSync.syncObservation() for each imported row, making them visible to MCP search(). (#1914) - Fix session-init cwd fallback to match context.ts (process.cwd()). Prevents project key mismatch that caused "no previous sessions" on fresh sessions. (#1918) - Fix sync-marketplace restart to include auth token and per-user port. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve all CodeRabbit and Greptile review comments on PR #2080 - Fix run.sh comment mismatch (no-op flag vs empty array) - Gate session-init on health check success (prevent running when worker unreachable) - Fix date_desc ordering ignored in FTS session search - Age-scope failed message purge (1h retention) instead of clearing all - Anchor RestartGuard decay to real successes (null init, not Date.now()) - Add recordSuccess() calls in ResponseProcessor and completion path - Prevent caller headers from overriding bearer auth token - Add lazy cleanup for rate limiter map to prevent unbounded growth - Bound post-import Chroma sync with concurrency limit of 8 - Add doc_type:'observation' filter to Chroma queries feeding observation hydration - Add FTS fallback to all specialized search handlers (observations, sessions, prompts, timeline) - Add response.ok check and error handling in viewer saveSettings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve CodeRabbit round-2 review comments - Use failure timestamp (COALESCE) instead of created_at_epoch for stale purge - Downgrade _fts5Available flag when FTS table creation fails - Escape FTS5 MATCH input by quoting user queries as literal phrases - Escape LIKE metacharacters (%, _, \) in prompt text search - Add response.ok check in initial settings load (matches save flow) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve CodeRabbit round-3 review comments - Include failed_at_epoch in COALESCE for age-scoped purge - Re-throw FTS5 errors so callers can distinguish failure from no-results - Wrap all FTS fallback calls in SearchManager with try/catch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove bearer auth and platform_source from context inject Bearer token auth (#1932/#1933) added friction for all localhost API clients with no benefit — the worker already binds localhost-only (CORS restriction + host binding). Removed auth-token module, requireAuth middleware, and Authorization headers from all internal callers. platform_source filtering from the /api/context/inject path was never used by any caller and silently filtered out observations. The underlying platform_source column stays; only the query-time filter and its plumbing through ContextBuilder, ObservationCompiler, SearchRoutes, context.ts, and transcripts/processor.ts are removed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: resolve CodeRabbit + Greptile + claude-review comments on PR #2081 - middleware.ts: drop 'Authorization' from CORS allowedHeaders (Greptile) - middleware.ts: rate limiter falls back to req.socket.remoteAddress; add Retry-After on 429 (claude-review) - SearchRoutes.ts: drop leftover platformSource read+pass in handleContextPreview (Greptile) - .docker-blowout-data/: stop tracking the empty SQLite placeholder and gitignore the dir (claude-review) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: tighten rate limiter — correct boundary + drop dead cleanup branch - `entry.count >= RATE_LIMIT_MAX_REQUESTS` so the 300th request is the first rejected (was 301). - Removed the `requestCounts.size > 100` lazy-cleanup block — on a localhost-only server the map tops out at 1–2 entries, so the branch was dead code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: rate limiter correctly allows exactly 300 req/min; doc localhost scope - Check `entry.count >= max` BEFORE incrementing so the cap matches the comment: 300 requests pass, the 301st gets 429. - Added a comment noting the limiter is effectively a global cap on a localhost-only worker (all callers share the 127.0.0.1/::1 bucket). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: normalise IPv4-mapped IPv6 in rate limiter client IP Strip the `::ffff:` prefix so a localhost caller routed as `::ffff:127.0.0.1` shares a bucket with `127.0.0.1`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: size-guarded prune of rate limiter map for non-localhost deploys Prune expired entries only when the map exceeds 1000 keys and we're already doing a window reset, so the cost is zero on the localhost hot path (1–2 keys) and the map can't grow unbounded if the worker is ever bound on a non-loopback interface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-20 13:31:13 -07:00
Alex Newman	8d166b47c1	Revert "revert: roll back v12.3.3 (Issue Blowout 2026)" ... This reverts commit bfc7de377a.	2026-04-20 12:18:55 -07:00
Alex Newman	bfc7de377a	revert: roll back v12.3.3 (Issue Blowout 2026) ... SessionStart context injection regressed in v12.3.3 — no memory context is being delivered to new sessions. Rolling back to the v12.3.2 tree state while the regression is investigated. Reverts #2080. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-20 11:59:15 -07:00
Alex Newman	ba1ef6c42c	fix: Issue Blowout 2026 — 25 bugs across worker, hooks, security, and search (#2080) ... * fix: resolve search, database, and docker bugs (#1913, #1916, #1956, #1957, #2048) - Fix concept/concepts param mismatch in SearchManager.normalizeParams (#1916) - Add FTS5 keyword fallback when ChromaDB is unavailable (#1913, #2048) - Add periodic WAL checkpoint and journal_size_limit to prevent unbounded WAL growth (#1956) - Add periodic clearFailed() to purge stale pending_messages (#1957) - Fix nounset-safe TTY_ARGS expansion in docker/claude-mem/run.sh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: prevent silent data loss on non-XML responses, add queue info to /health (#1867, #1874) - ResponseProcessor: mark messages as failed (with retry) instead of confirming when the LLM returns non-XML garbage (auth errors, rate limits) (#1874) - Health endpoint: include activeSessions count for queue liveness monitoring (#1867) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: cache isFts5Available() at construction time Addresses Greptile review: avoid DDL probe (CREATE + DROP) on every text query. Result is now cached in _fts5Available at construction. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve worker stability bugs — pool deadlock, MCP loopback, restart guard (#1868, #1876, #2053) - Replace flat consecutiveRestarts counter with time-windowed RestartGuard: only counts restarts within 60s window (cap=10), decays after 5min of success. Prevents stranding pending messages on long-running sessions. (#2053) - Add idle session eviction to pool slot allocation: when all slots are full, evict the idlest session (no pending work, oldest activity) to free a slot for new requests, preventing 60s timeout deadlock. (#1868) - Fix MCP loopback self-check: use process.execPath instead of bare 'node' which fails on non-interactive PATH. Fix crash misclassification by removing false "Generator exited unexpectedly" error log on normal completion. (#1876) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve hooks reliability bugs — summarize exit code, session-init health wait (#1896, #1901, #1903, #1907) - Wrap summarize hook's workerHttpRequest in try/catch to prevent exit code 2 (blocking error) on network failures or malformed responses. Session exit no longer blocks on worker errors. (#1901) - Add health-check wait loop to UserPromptSubmit session-init command in hooks.json. On Linux/WSL where hook ordering fires UserPromptSubmit before SessionStart, session-init now waits up to 10s for worker health before proceeding. Also wrap session-init HTTP call in try/catch. (#1907) - Close #1896 as already-fixed: mtime comparison at file-context.ts:255-267 bypasses truncation when file is newer than latest observation. - Close #1903 as no-repro: hooks.json correctly declares all hook events. Issue was Claude Code 12.0.1/macOS platform event-dispatch bug. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: security hardening — bearer auth, path validation, rate limits, per-user port (#1932, #1933, #1934, #1935, #1936) - Add bearer token auth to all API endpoints: auto-generated 32-byte token stored at ~/.claude-mem/worker-auth-token (mode 0600). All hook, MCP, viewer, and OpenCode requests include Authorization header. Health/readiness endpoints exempt for polling. (#1932, #1933) - Add path traversal protection: watch.context.path validated against project root and ~/.claude-mem/ before write. Rejects ../../../etc style attacks. (#1934) - Reduce JSON body limit from 50MB to 5MB. Add in-memory rate limiter (300 req/min/IP) to prevent abuse. (#1935) - Derive default worker port from UID (37700 + uid%100) to prevent cross-user data leakage on multi-user macOS. Windows falls back to 37777. Shell hooks use same formula via id -u. (#1936) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve search project filtering and import Chroma sync (#1911, #1912, #1914, #1918) - Fix per-type search endpoints to pass project filter to Chroma queries and SQLite hydration. searchObservations/Sessions/UserPrompts now use $or clause matching project + merged_into_project. (#1912) - Fix timeline/search methods to pass project to Chroma anchor queries. Prevents cross-project result leakage when project param omitted. (#1911) - Sync imported observations to ChromaDB after FTS rebuild. Import endpoint now calls chromaSync.syncObservation() for each imported row, making them visible to MCP search(). (#1914) - Fix session-init cwd fallback to match context.ts (process.cwd()). Prevents project key mismatch that caused "no previous sessions" on fresh sessions. (#1918) - Fix sync-marketplace restart to include auth token and per-user port. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve all CodeRabbit and Greptile review comments on PR #2080 - Fix run.sh comment mismatch (no-op flag vs empty array) - Gate session-init on health check success (prevent running when worker unreachable) - Fix date_desc ordering ignored in FTS session search - Age-scope failed message purge (1h retention) instead of clearing all - Anchor RestartGuard decay to real successes (null init, not Date.now()) - Add recordSuccess() calls in ResponseProcessor and completion path - Prevent caller headers from overriding bearer auth token - Add lazy cleanup for rate limiter map to prevent unbounded growth - Bound post-import Chroma sync with concurrency limit of 8 - Add doc_type:'observation' filter to Chroma queries feeding observation hydration - Add FTS fallback to all specialized search handlers (observations, sessions, prompts, timeline) - Add response.ok check and error handling in viewer saveSettings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve CodeRabbit round-2 review comments - Use failure timestamp (COALESCE) instead of created_at_epoch for stale purge - Downgrade _fts5Available flag when FTS table creation fails - Escape FTS5 MATCH input by quoting user queries as literal phrases - Escape LIKE metacharacters (%, _, \) in prompt text search - Add response.ok check in initial settings load (matches save flow) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve CodeRabbit round-3 review comments - Include failed_at_epoch in COALESCE for age-scoped purge - Re-throw FTS5 errors so callers can distinguish failure from no-results - Wrap all FTS fallback calls in SearchManager with try/catch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-20 11:42:09 -07:00
Alex Newman	be99a5d690	fix: resolve search, database, and docker bugs (#2079) ... * fix: resolve search, database, and docker bugs (#1913, #1916, #1956, #1957, #2048) - Fix concept/concepts param mismatch in SearchManager.normalizeParams (#1916) - Add FTS5 keyword fallback when ChromaDB is unavailable (#1913, #2048) - Add periodic WAL checkpoint and journal_size_limit to prevent unbounded WAL growth (#1956) - Add periodic clearFailed() to purge stale pending_messages (#1957) - Fix nounset-safe TTY_ARGS expansion in docker/claude-mem/run.sh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: prevent silent data loss on non-XML responses, add queue info to /health (#1867, #1874) - ResponseProcessor: mark messages as failed (with retry) instead of confirming when the LLM returns non-XML garbage (auth errors, rate limits) (#1874) - Health endpoint: include activeSessions count for queue liveness monitoring (#1867) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: cache isFts5Available() at construction time Addresses Greptile review: avoid DDL probe (CREATE + DROP) on every text query. Result is now cached in _fts5Available at construction. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-19 22:19:18 -07:00
Alex Newman	f467763340	chore: bump version to 12.3.1 ... Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-19 20:35:01 -07:00
Alex Newman	789efe4234	feat: disable subagent summaries, label subagent observations (#2073) ... * feat: disable subagent summaries and label subagent observations Detect Claude Code subagent hook context via `agent_id`/`agent_type` on stdin, short-circuit the Stop-hook summary path when present, and thread the subagent identity end-to-end onto observation rows (new `agent_type` and `agent_id` columns, migration 010 at version 27). Main-session rows remain NULL; content-hash dedup is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address PR #2073 review feedback - Narrow summarize subagent guard to agentId only so --agent-started main sessions still own their summary (agentType alone is main-session). - Remove now-dead agentId/agentType spreads from the summarize POST body. - Always overwrite pendingAgentId/pendingAgentType in SDK/Gemini/OpenRouter agents (clears stale subagent identity on main-session messages after a subagent message in the same batch). - Add idx_observations_agent_id index in migration 010 + the mirror migration in SessionStore + the runner. - Replace console.log in migration010 with logger.debug. - Update summarize test: agentType alone no longer short-circuits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit + claude-review iteration 4 feedback - SessionRoutes.handleSummarizeByClaudeId: narrow worker-side guard to agentId only (matches hook-side). agentType alone = --agent main session, which still owns its summary. - ResponseProcessor: wrap storeObservations in try/finally so pendingAgentId/Type clear even if storage throws. Prevents stale subagent identity from leaking into the next batch on error. - SessionStore.importObservation + bulk.importObservation: persist agent_type/agent_id so backup/import round-trips preserve subagent attribution. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * polish: claude-review iteration 5 cleanup - Use ?? not || for nullable subagent fields in PendingMessageStore (prevents treating empty string as null). - Simplify observation.ts body spread — include fields unconditionally; JSON.stringify drops undefined anyway. - Narrow any[] to Array<{ name: string }> in migration010 column checks. - Add trailing newline to migrations.ts. - Document in observations/store.ts why the dedup hash intentionally excludes agent fields. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * polish: claude-review iteration 7 feedback - claude-code adapter: add 128-char safety cap on agent_id/agent_type so a malformed Claude Code payload cannot balloon DB rows. Empty strings now also treated as absent. - migration010: state-aware debug log lists only columns actually added; idempotent re-runs log "already present; ensured indexes". - Add 3 adapter tests covering the length cap boundary and empty-string rejection. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf: skip subagent summary before worker bootstrap Move the agentId short-circuit above ensureWorkerRunning() so a Stop hook fired inside a subagent does not trigger worker startup just to return early. Addresses CodeRabbit nit on summarize.ts:36-47. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-19 14:58:01 -07:00
Alex Newman	7a66cb310f	fix(worktree): address PR review — schema guard, startup adoption, query parity ... Addresses six CodeRabbit/Greptile findings on PR #2052: - Schema guard in adoptMergedWorktrees probes for merged_into_project columns before preparing statements; returns early when absent so first boot after upgrade (pre-migration) doesn't silently fail. - Startup adoption now iterates distinct cwds from pending_messages and dedupes via resolveMainRepoPath — the worker daemon runs with cwd=plugin scripts dir, so process.cwd() fallback was a no-op. - ObservationCompiler single-project queries (queryObservations / querySummaries) OR merged_into_project into WHERE so injected context surfaces adopted worktree rows, matching the Multi variants. - SessionStore constructor now calls ensureMergedIntoProjectColumns so bundled artifacts (context-generator.cjs) that embed SessionStore get the merged_into_project column on DBs that only went through the bundled migration chain. - OBSERVER_SESSIONS_PROJECT constant is now derived from basename(OBSERVER_SESSIONS_DIR) and used across PaginationHelper, SessionStore, and timeline queries instead of hardcoded strings. - Corrected misleading Chroma retry docstring in WorktreeAdoption to match actual behavior (no auto-retry once SQL commits).	2026-04-16 21:31:30 -07:00
Alex Newman	d1601123fd	feat(ui): hide observer-sessions project from UI lists ... Observer sessions (internal SDK-driven worker queries) run under a synthetic project name 'observer-sessions' to keep them out of claude --resume. They were still surfacing in the viewer project picker and unfiltered observation/summary/prompt feeds. Filter them out at every UI-facing query: - SessionStore.getAllProjects and getProjectCatalog - timeline/queries.ts getAllProjects - PaginationHelper observations/summaries/prompts when no project is selected When a caller explicitly requests project='observer-sessions', results are still returned (not a hard ban, just hidden by default). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-16 20:05:37 -07:00
Alex Newman	f6fda8fff4	fix(worktree): address CodeRabbit PR review feedback ... - Document --branch override in npx-cli help text - Guard ContextBuilder against empty projects[] override; fall back to cwd-derived primary - Ensure merged_into_project indexes are created even if ALTER ran in a prior partial migration - Reject adopt --branch/--cwd flags with missing or flag-like values - Use defined --color-border-primary token for merged badge border Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-16 20:03:27 -07:00
Alex Newman	0a5f072aaf	build(worktree): rebuild plugin artifacts for worktree adoption feature ... Regenerated worker-service.cjs, context-generator.cjs, viewer.html, and viewer-bundle.js to reflect all six implementation phases of the merged- worktree adoption feature. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-16 19:36:00 -07:00
Alex Newman	dc198d5677	feat(worktree): include parent repo observations in worktree read scope ... Worktrees are branches off main; the parent holds the architecture, decisions, and long-tail history the worktree inherits. Scoping reads to the worktree alone meant every new worktree started cold on any question that required prior context. Expand `allProjects` in a worktree to `[parent, composite]` so reads pull both. Writes still go through `.primary` (the composite), so sibling worktrees don't leak into each other. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-16 17:55:17 -07:00
Alex Newman	a65ab055ca	fix(worktree): audit observation fetch/display for composite project names ... Three sites didn't account for parent/worktree composite naming: - PaginationHelper.stripProjectPath: marker used full composite, breaking path sanitization for worktrees checked out outside a parent/leaf layout. Now extracts the leaf segment. - observations/store.ts: fallback imported getCurrentProjectName from shared/paths.ts (a duplicate impl without worktree detection). Switched to getProjectContext().primary so writes key into the same project as reads. - SearchManager.getRecentContext: fallback used basename(cwd) and lost the parent prefix, making the MCP tool find nothing in worktrees. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-16 17:38:49 -07:00
Alex Newman	3869b083d0	fix(context): derive project from explicit projects array, not cwd ... When a caller (e.g. worker context-inject route) passes a `projects` array without a matching cwd, the cwd-derived `context.primary` drifted from the projects being queried — producing an empty-state header for one project while querying another. Use the last entry of `projects` so header and query target stay in sync.	2026-04-16 17:16:51 -07:00
Alex Newman	040729beef	fix(project-name): use parent/worktree composite so observations don't cross worktrees ... Revert of #1820 behavior. Each worktree now gets its own bucket: - In a worktree, primary = `parent/worktree` (e.g. `claude-mem/dar-es-salaam`) - In a main repo, primary = basename (unchanged) - allProjects is always `[primary]` — strict isolation at query time Includes a one-off maintenance script (scripts/worktree-remap.ts) that retroactively reattributes past sessions to their worktree using path signals in observations and user prompts. Two-rule inference keeps the remap high-confidence: 1. The worktree basename in the path matches the session's current plain project name (pre-#1820 era; trusted). 2. Or all worktree path signals converge on a single (parent, worktree) across the session. Ambiguous sessions are skipped. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-16 15:40:44 -07:00
Alex Newman	4c792f026d	build: rebuild plugin artifacts after $CMEM header revert ... Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-15 12:05:49 -07:00
Alex Newman	d0fc68c630	revert: remove overengineered summary salvage logic (#1718) (#1850) ... The synthetic summary salvage feature created fake summaries from observation data when the AI returned <observation> instead of <summary> tags. This was overengineered — missing a summary is preferable to fabricating one from observation fields that don't map cleanly to summary semantics. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-15 04:22:41 -07:00
Jochen Meyer	59ce0fc553	fix: exclude primary key index from unique constraint check in migration 7 (#1771) ... * fix: exclude primary key index from unique constraint check in migration 7 PRAGMA index_list returns all indexes including those backing PRIMARY KEY columns (origin='pk'), which always have unique=1. The check was therefore always true, causing migration 7 to run the full DROP/CREATE/RENAME table sequence on every worker startup instead of short-circuiting once the UNIQUE constraint had already been removed. Fix: filter to non-PK indexes by requiring idx.origin !== 'pk'. The origin field is already present on the existing IndexInfo interface. Fixes #1749 * fix: apply pk-origin guard to all three migration code paths CodeRabbit correctly identified that the origin !== 'pk' fix was only applied to MigrationRunner.ts but not to the two other active code paths that run the same removeSessionSummariesUniqueConstraint logic: - SessionStore.ts:220 — used by DatabaseManager and worker-service - plugin/scripts/context-generator.cjs — bundled artifact (minified) All three paths now consistently exclude primary-key indexes when detecting unique constraints on session_summaries.	2026-04-15 00:57:51 -07:00
Alex Newman	b701bf29e6	chore: bump version to 12.1.0 ... Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-08 17:31:19 -07:00
Alex Newman	f4570f2a0a	chore: rebuild plugin artifacts ... Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-07 12:31:53 -07:00
huakson	4f6fb9e614	fix: address platform source review feedback ... Tighten platform source persistence so legacy callers cannot silently relabel existing sessions, repair migration 24 when schema_versions drifts from the real schema, and polish the follow-up UI/error-handler review nits. - only backfill platform_source when it is blank and raise on explicit source conflicts for an existing session - make migration 24 verify both the sdk_sessions column and its index before treating it as applied - expose platform_source from the functional session getters and add regression tests for source preservation and schema drift recovery - add the required APPROVED OVERRIDE annotation for centralized HTTP error translation - keep mobile source pills on a single horizontal row	2026-03-24 10:46:48 -03:00
huakson	2b60dd2932	feat: isolate Claude and Codex session sources ... Persist platform_source across session creation, transcript ingestion, API query paths, and viewer state so Claude and Codex data can coexist without bleeding into each other. - add platform-source normalization helpers and persist platform_source in sdk_sessions via migration 24 with backfill and indexing - thread platformSource through CLI hooks, transcript processing, context generation, pagination, search routes, SSE payloads, and session management - expose source-aware project catalogs, viewer tabs, context preview selectors, and source badges for observations, prompts, and summaries - start the transcript watcher from the worker for transcript-based clients and preserve platform source during Codex ingestion - auto-start the worker from the MCP server for MCP-only clients and tighten stdio-driven cleanup during shutdown - keep createSDKSession backward compatible with existing custom-title callers while allowing explicit platform source forwarding	2026-03-24 08:46:18 -03:00
Alex Newman	d54e574251	chore: bump version to 10.6.1 ... Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-18 14:36:23 -07:00
Alex Newman	7e07210635	feat: add timeline-report skill with token economics, compress context output 53% ... ## Summary - New timeline-report skill for generating narrative project history reports - Compressed markdown context output ~53% (tables → flat compact lines, verbose labels → terse format) - Added `full=true` param to /api/context/inject for fetching all observations - Split TimelineRenderer into separate markdown/color rendering paths - Removed arbitrary file write vulnerability (dump_to_file param) - Fixed timestamp ditto marker leaking across session summary boundaries ## Review - Rebased on main (v10.6.0) to preserve OpenClaw system prompt injection - Reviewed by /review (gstack) + /octo:review (Codex, Gemini, Claude fleet) - Security fix (dump_to_file removal) confirmed by all 3 reviewers - Timestamp bug caught by Codex, fixed 🤖 Generated with [Claude Code](https://claude.com/claude-code)	2026-03-18 13:57:20 -07:00
Alex Newman	80a8c90a1a	feat: add embedded Process Supervisor for unified process lifecycle (#1370) ... * feat: add embedded Process Supervisor for unified process lifecycle management Consolidates scattered process management (ProcessManager, GracefulShutdown, HealthMonitor, ProcessRegistry) into a unified src/supervisor/ module. New: ProcessRegistry with JSON persistence, env sanitizer (strips CLAUDECODE_* vars), graceful shutdown cascade (SIGTERM → 5s wait → SIGKILL with tree-kill on Windows), PID file liveness validation, and singleton Supervisor API. Fixes #1352 (worker inherits CLAUDECODE env causing nested sessions) Fixes #1356 (zombie TCP socket after Windows reboot) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add session-scoped process reaping to supervisor Adds reapSession(sessionId) to ProcessRegistry for killing session-tagged processes on session end. SessionManager.deleteSession() now triggers reaping. Tightens orphan reaper interval from 60s to 30s. Fixes #1351 (MCP server processes leak on session end) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add Unix domain socket support for worker communication Introduces socket-manager.ts for UDS-based worker communication, eliminating port 37777 collisions between concurrent sessions. Worker listens on ~/.claude-mem/sockets/worker.sock by default with TCP fallback. All hook handlers, MCP server, health checks, and admin commands updated to use socket-aware workerHttpRequest(). Backwards compatible — settings can force TCP mode via CLAUDE_MEM_WORKER_TRANSPORT=tcp. Fixes #1346 (port 37777 collision across concurrent sessions) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove in-process worker fallback from hook command Removes the fallback path where hook scripts started WorkerService in-process, making the worker a grandchild of Claude Code (killed by sandbox). Hooks now always delegate to ensureWorkerStarted() which spawns a fully detached daemon. Fixes #1249 (grandchild process killed by sandbox) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add health checker and /api/admin/doctor endpoint Adds 30-second periodic health sweep that prunes dead processes from the supervisor registry and cleans stale socket files. Adds /api/admin/doctor endpoint exposing supervisor state, process liveness, and environment health. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add comprehensive supervisor test suite 64 tests covering all supervisor modules: process registry (18 tests), env sanitizer (8), shutdown cascade (10), socket manager (15), health checker (5), and supervisor API (6). Includes persistence, isolation, edge cases, and cross-module integration scenarios. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: revert Unix domain socket transport, restore TCP on port 37777 The socket-manager introduced UDS as default transport, but this broke the HTTP server's TCP accessibility (viewer UI, curl, external monitoring). Since there's only ever one worker process handling all sessions, the port collision rationale for UDS doesn't apply. Reverts to TCP-only, removing ~900 lines of unnecessary complexity. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove dead code found in pre-landing review Remove unused `acceptingSpawns` field from Supervisor class (written but never read — assertCanSpawn uses stopPromise instead) and unused `buildWorkerUrl` import from context handler. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * updated gitignore * fix: address PR review feedback - downgrade HTTP logging, clean up gitignore, harden supervisor - Downgrade request/response HTTP logging from info to debug to reduce noise - Remove unused getWorkerPort imports, use buildWorkerUrl helper - Export ENV_PREFIXES/ENV_EXACT_MATCHES from env-sanitizer, reuse in Server.ts - Fix isPidAlive(0) returning true (should be false) - Add shutdownInitiated flag to prevent signal handler race condition - Make validateWorkerPidFile testable with pidFilePath option - Remove unused dataDir from ShutdownCascadeOptions - Upgrade reapSession log from debug to warn - Rename zombiePidFiles to deadProcessPids (returns actual PIDs) - Clean up gitignore: remove duplicate datasets/, stale ~*/ and http*/ patterns - Fix tests to use temp directories instead of relying on real PID file Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-16 14:49:23 -07:00
Alex Newman	6581d2ef45	fix: unify mode type/concept loading to always use mode definition (#1316) ... * fix: unify mode type/concept loading to always use mode definition Code mode previously read observation types/concepts from settings.json while non-code modes read from their mode JSON definition. This caused stale filters to persist when switching between modes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: remove dead observation type/concept settings constants CLAUDE_MEM_CONTEXT_OBSERVATION_TYPES and OBSERVATION_CONCEPTS are no longer read by ContextConfigLoader since all modes now use their mode definition. Removes the constants, defaults, UI controls, and the now-empty observation-metadata.ts file. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-09 03:00:20 -07:00
Alex Newman	4144010264	chore: bump version to 10.4.1	2026-02-23 22:13:11 -05:00
Alex Newman	23f30d35b9	chore: bump version to 10.4.0 ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-23 19:39:06 -05:00
Alex Newman	c6f932988a	Fix 30+ root-cause bugs across 10 triage phases (#1214) ... * MAESTRO: fix ChromaDB core issues — Python pinning, Windows paths, disable toggle, metadata sanitization, transport errors - Add --python version pinning to uvx args in both local and remote mode (fixes #1196, #1206, #1208) - Convert backslash paths to forward slashes for --data-dir on Windows (fixes #1199) - Add CLAUDE_MEM_CHROMA_ENABLED setting for SQLite-only fallback mode (fixes #707) - Sanitize metadata in addDocuments() to filter null/undefined/empty values (fixes #1183, #1188) - Wrap callTool() in try/catch for transport errors with auto-reconnect (fixes #1162) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix data integrity — content-hash deduplication, project name collision, empty project guard, stuck isProcessing - Add SHA-256 content-hash deduplication to observations INSERT (store.ts, transactions.ts, SessionStore.ts) - Add content_hash column via migration 22 with backfill and index - Fix project name collision: getCurrentProjectName() now returns parent/basename - Guard against empty project string with cwd-derived fallback - Fix stuck isProcessing: hasAnyPendingWork() resets processing messages older than 5 minutes - Add 12 new tests covering all four fixes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix hook lifecycle — stderr suppression, output isolation, conversation pollution prevention - Suppress process.stderr.write in hookCommand() to prevent Claude Code showing diagnostic output as error UI (#1181). Restores stderr in finally block for worker-continues case. - Convert console.error() to logger.warn()/error() in hook-command.ts and handlers/index.ts so all diagnostics route to log file instead of stderr. - Verified all 7 handlers return suppressOutput: true (prevents conversation pollution #598, #784). - Verified session-complete is a recognized event type (fixes #984). - Verified unknown event types return no-op handler with exit 0 (graceful degradation). - Added 10 new tests in tests/hook-lifecycle.test.ts covering event dispatch, adapter defaults, stderr suppression, and standard response constants. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix worker lifecycle — restart loop coordination, stale transport retry, ENOENT shutdown race - Add PID file mtime guard to prevent concurrent restart storms (#1145): isPidFileRecent() + touchPidFile() coordinate across sessions - Add transparent retry in ChromaMcpManager.callTool() on transport error — reconnects and retries once instead of failing (#1131) - Wrap getInstalledPluginVersion() with ENOENT/EBUSY handling (#1042) - Verified ChromaMcpManager.stop() already called on all shutdown paths Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix Windows platform support — uvx.cmd spawn, PowerShell $_ elimination, windowsHide, FTS5 fallback - Route uvx spawn through cmd.exe /c on Windows since MCP SDK lacks shell:true (#1190, #1192, #1199) - Replace all PowerShell Where-Object {$_} pipelines with WQL -Filter server-side filtering (#1024, #1062) - Add windowsHide: true to all exec/spawn calls missing it to prevent console popups (#1048) - Add FTS5 runtime probe with graceful fallback when unavailable on Windows (#791) - Guard FTS5 table creation in migrations, SessionSearch, and SessionStore with try/catch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix skills/ distribution — build-time verification and regression tests (#1187) Add post-build verification in build-hooks.js that fails if critical distribution files (skills, hooks, plugin manifest) are missing. Add 10 regression tests covering skill file presence, YAML frontmatter, hooks.json integrity, and package.json files field. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix MigrationRunner schema initialization (#979) — version conflict between parallel migration systems Root cause: old DatabaseManager migrations 1-7 shared schema_versions table with MigrationRunner's 4-22, causing version number collisions (5=drop tables vs add column, 6=FTS5 vs prompt tracking, 7=discovery_tokens vs remove UNIQUE). initializeSchema() was gated behind maxApplied===0, so core tables were never created when old versions were present. Fixes: - initializeSchema() always creates core tables via CREATE TABLE IF NOT EXISTS - Migrations 5-7 check actual DB state (columns/constraints) not just version tracking - Crash-safe temp table rebuilds (DROP IF EXISTS _new before CREATE) - Added missing migration 21 (ON UPDATE CASCADE) to MigrationRunner - Added ON UPDATE CASCADE to FK definitions in initializeSchema() - All changes applied to both runner.ts and SessionStore.ts Tests: 13 new tests in migration-runner.test.ts covering fresh DB, idempotency, version conflicts, crash recovery, FK constraints, and data integrity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix 21 test failures — stale mocks, outdated assertions, missing OpenClaw guards Server tests (12): Added missing workerPath and getAiStatus to ServerOptions mocks after interface expansion. ChromaSync tests (3): Updated to verify transport cleanup in ChromaMcpManager after architecture refactor. OpenClaw (2): Added memory_ tool skipping and response truncation to prevent recursive loops and oversized payloads. MarkdownFormatter (2): Updated assertions to match current output. SettingsDefaultsManager (1): Used correct default key for getBool test. Logger standards (1): Excluded CLI transcript command from background service check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix Codex CLI compatibility (#744) — session_id fallbacks, unknown platform tolerance, undefined guard Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix Cursor IDE integration (#838, #1049) — adapter field fallbacks, tolerant session-init validation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix /api/logs OOM (#1203) — tail-read replaces full-file readFileSync Replace readFileSync (loads entire file into memory) with readLastLines() that reads only from the end of the file in expanding chunks (64KB → 10MB cap). Prevents OOM on large log files while preserving the same API response shape. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix Settings CORS error (#1029) — explicit methods and allowedHeaders in CORS config Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: add session custom_title for agent attribution (#1213) — migration 23, endpoint + store support Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: prevent CLAUDE.md/AGENTS.md writes inside .git/ directories (#1165) Add .git path guard to all 4 write sites to prevent ref corruption when paths resolve inside .git internals. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix plugin disabled state not respected (#781) — early exit check in all hook entry points Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix UserPromptSubmit context re-injection on every turn (#1079) — contextInjected session flag Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix stale AbortController queue stall (#1099) — lastGeneratorActivity tracking + 30s timeout Three-layer fix: 1. Added lastGeneratorActivity timestamp to ActiveSession, updated by processAgentResponse (all agents), getMessageIterator (queue yields), and startGeneratorWithProvider (generator launch) 2. Added stale generator detection in ensureGeneratorRunning — if no activity for >30s, aborts stale controller, resets state, restarts 3. Added AbortSignal.timeout(30000) in deleteSession to prevent indefinite hang when awaiting a stuck generator promise Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-23 19:34:35 -05:00
Alex Newman	2db9d0e383	chore: bump version to 10.3.3 ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-23 03:47:48 -05:00
Alex Newman	7966c6cba9	fix: rename save_memory and fix MCP search instructions + startup hook (#1210) ... * fix: rename save_memory to save_observation and fix MCP search instructions Stop the primary agent from proactively saving memories by renaming save_memory to save_observation with a neutral description. Remove "Saving Memories" section from SKILL.md. Update context formatters and output styles to reference the mem-search skill instead of raw MCP tool names. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: split SessionStart hooks so smart-install failure doesn't block worker start smart-install.js and worker-start were in the same hook group, so if smart-install exited non-zero the worker never started. Split into separate hook groups so they run independently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: worker startup waits for readiness before hooks fire Move initializationCompleteFlag to set after DB/search init (not MCP), add waitForReadiness() polling /api/readiness, and extract shared pollEndpointUntilOk helper to DRY up health/readiness checks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-23 03:30:31 -05:00
Alex Newman	40daf8f3fa	feat: replace WASM embeddings with persistent chroma-mcp MCP connection (#1176) ... * feat: replace WASM embeddings with persistent chroma-mcp MCP connection Replace ChromaServerManager (npx chroma run + chromadb npm + ONNX/WASM) with ChromaMcpManager, a singleton stdio MCP client that communicates with chroma-mcp via uvx. This eliminates native binary issues, segfaults, and WASM embedding failures that plagued cross-platform installs. Key changes: - Add ChromaMcpManager: singleton MCP client with lazy connect, auto-reconnect, connection lock, and Zscaler SSL cert support - Rewrite ChromaSync to use MCP tool calls instead of chromadb npm client - Handle chroma-mcp's non-JSON responses (plain text success/error messages) - Treat "collection already exists" as idempotent success - Wire ChromaMcpManager into GracefulShutdown for clean subprocess teardown - Delete ChromaServerManager (no longer needed) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — connection guard leak, timer leak, async reset - Clear connecting guard in finally block to prevent permanent reconnection block - Clear timeout after successful connection to prevent timer leak - Make reset() async to await stop() before nullifying instance - Delete obsolete chroma-server-manager test (imports deleted class) - Update graceful-shutdown test to use chromaMcpManager property name Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: prevent chroma-mcp spawn storm — zombie cleanup, stale onclose guard, reconnect backoff Three bugs caused chroma-mcp processes to accumulate (92+ observed): 1. Zombie on timeout: failed connections left subprocess alive because only the timer was cleared, not the transport. Now catch block explicitly closes transport+client before rethrowing. 2. Stale onclose race: old transport's onclose handler captured `this` and overwrote the current connection reference after reconnect, orphaning the new subprocess. Now guarded with reference check. 3. No backoff: every failure triggered immediate reconnect. With backfill doing hundreds of MCP calls, this created rapid-fire spawning. Added 10s backoff on both connection failure and unexpected process death. Also includes ChromaSync fixes from PR review: - queryChroma deduplication now preserves index-aligned arrays - SQL injection guard on backfill ID exclusion lists Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-18 18:32:38 -05:00
Alex Newman	b88251bc8b	fix: self-healing claimNextMessage prevents stuck processing messages (#1159) ... * fix: self-healing claimNextMessage prevents stuck processing messages claimAndDelete → claimNextMessage with atomic self-healing: resets stale processing messages (>60s) back to pending before claiming. Eliminates stuck messages from generator crashes without external timers. Removes redundant idle-timeout reset in worker-service.ts. Adds QUEUE to logger Component type. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: update stale comments in SessionQueueProcessor to reflect claim-confirm pattern Comments still referenced the old claim-and-delete pattern after the claimNextMessage rename. Updated to accurately describe the current lifecycle where messages are marked as processing and stay in DB until confirmProcessed() is called. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: move Date.now() inside transaction and extract stale threshold constant - Move Date.now() inside claimNextMessage transaction closure so timestamp is fresh if WAL contention causes retry - Extract STALE_PROCESSING_THRESHOLD_MS to module-level constant - Add comment clarifying strict < boundary semantics Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-17 23:15:46 -05:00
Alex Newman	854bf922a4	chore: bump version to 10.2.0 ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-16 16:40:10 -05:00
Alex Newman	34358ab33d	feat: add systemMessage support for SessionStart hook and tune defaults ... Add systemMessage field to HookResult so SessionStart can display a colored timeline directly to the user in the CLI. The handler now parallel-fetches both markdown (for Claude context) and ANSI-colored (for user display) timelines, appending a viewer URL link. Also update default settings to hide verbose token columns (read/work tokens, savings amount) and disable full observation expansion, keeping the cleaner index-only view by default. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-16 00:05:13 -05:00
Alex Newman	1b68c55763	fix: resolve SDK spawn failures and sharp native binary crashes ... - Strip CLAUDECODE env var from SDK subprocesses to prevent "cannot be launched inside another Claude Code session" error (Claude Code 2.1.42+) - Lazy-load @chroma-core/default-embed to avoid eagerly pulling in sharp native binaries at bundle startup (fixes ERR_DLOPEN_FAILED) - Add stderr capture to SDK spawn for diagnosing future process failures - Exclude lockfiles from marketplace rsync and delete stale lockfiles before npm install to prevent native dep version mismatches Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-13 22:47:27 -05:00