From 9e2973059aa650c1be03d270584a6fe7c15da051 Mon Sep 17 00:00:00 2001 From: Alex Newman Date: Sat, 2 May 2026 16:05:56 -0700 Subject: [PATCH] UX redesign: installer + provider rename + /learn-codebase + welcome card + SessionStart hint (#2255) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(ux): claude-mem UX improvements with installer enhancements Squashed PR #2156 commits for clean rebase onto main: - feat(installer): add provider selection, model prompt, worker auto-start - refactor: rename *Agent provider classes to *Provider - feat: add /learn-codebase skill and viewer welcome card - feat(worker): inject welcome hint when project has zero observations - fix(pr-2156): address greptile review comments - fix(pr-2156): address coderabbit review comments - fix(pr-2156): persist CLAUDE_MEM_PROVIDER for non-claude in non-TTY mode - fix(pr-2156): file-backed settings reads in installer + env-first SKILL doc Co-Authored-By: Claude Opus 4.7 (1M context) * build: rebuild plugin artifacts after rebase onto v12.4.7 Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(skills): strip claude-mem internals from learn-codebase The learn-codebase skill, install next-step copy, WelcomeCard, and welcome-hint previously walked the primary agent through worker endpoints and synthetic observation payloads. The PostToolUse hook already captures every Read/Edit the agent makes — the agent should have no awareness that the memory layer exists. Collapse the skill to one instruction ("read every source file in full") and rephrase touchpoints to describe only what the user observes (Claude reading files), not what happens behind the scenes. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(sync): preflight version mismatch + settings-aware port resolution Two related fixes for build-and-sync's worker restart step: 1. Read CLAUDE_MEM_WORKER_PORT from ~/.claude-mem/settings.json the same way the worker does, instead of computing the default port from the uid alone. Previously, users with a custom port saw a misleading "Worker not running" message because the restart POST hit the wrong port and got ECONNREFUSED. 2. Add a preflight check that aborts the sync when the running worker's reported version does not match the version we are about to build. Claude Code's plugin loader pins the worker to a specific cache version per session, so syncing into a newer cache directory has no effect until the user runs `claude plugin update thedotmack/claude-mem` to bump the pin. The preflight surfaces this explicitly with the exact command to run; --force bypasses it for intentional cases. Co-Authored-By: Claude Opus 4.7 (1M context) * docs(learn-codebase): note sed for partial reads of large files Co-Authored-By: Claude Opus 4.7 (1M context) * refactor: strip comments codebase-wide Removed prose comments from all tracked source. Preserved directives (@ts-ignore, eslint-disable, biome-ignore, prettier-ignore, triple-slash references, webpack magic, shebangs). Deleted two tests that asserted on comment text rather than runtime behavior. Net: 401 files, -14,587 / +389 lines, -10.4% bytes. Verified: typecheck passes, build passes, test count unchanged from baseline (22 pre-existing fails, all unrelated). Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(installer): move runtime setup into npx, eliminate hook dead air Smart-install ran 3 times during a fresh install — the worst run was silent, fired by Claude Code's Setup hook after `claude plugin install`, producing ~30s of dead air that looked like the plugin was hung. This change makes `npx claude-mem install` the single place heavy work happens, with a visible spinner. Hooks become runtime-only. - New `src/npx-cli/install/setup-runtime.ts` module: ensureBun, ensureUv, installPluginDependencies, read/writeInstallMarker, isInstallCurrent. Marker schema preserved exactly ({version, bun, uv, installedAt}) so ContextBuilder and BranchManager readers keep working. - `npx claude-mem install`: ungated copy/register/enable for every IDE, inserts a "Setting up runtime" task with honest "first install can take ~30s" spinner. The claude-code shell-out to `claude plugin install` is removed — npx already populated everything Claude reads. - New `npx claude-mem repair` command for post-`claude plugin update` recovery, force-reinstalls runtime. - Setup hook now runs `plugin/scripts/version-check.js` (29ms wall) instead of smart-install. Mismatch prints "run: npx claude-mem repair" on stderr. Always exits 0 (non-blocking, per CLAUDE.md exit-code strategy). - SessionStart loses the smart-install entry; 2 hooks remain (worker start, context fetch). Co-Authored-By: Claude Opus 4.7 (1M context) * chore(installer): delete smart-install sources, retarget tests - Delete scripts/smart-install.js + plugin/scripts/smart-install.js (both are source files kept in sync manually; both must go). - Delete tests/smart-install.test.ts (covered surface is gone). - tests/plugin-scripts-line-endings: drop smart-install.js entry. - tests/infrastructure/plugin-distribution: retarget two assertions at version-check.js (the new Setup hook script). - New tests/setup-runtime.test.ts: 9 tests covering marker read/write, isInstallCurrent semantics. Marker schema invariant verified. Co-Authored-By: Claude Opus 4.7 (1M context) * docs(installer): describe npx-driven setup + version-check Setup hook Sweep public docs and architecture notes to reflect the new flow: npx installer does Bun/uv setup with a visible spinner; Setup hook runs sub-100ms version-check.js; users hit `npx claude-mem repair` after a `claude plugin update`. - docs/architecture-overview.md: hook lifecycle table + npx flow paragraph - docs/public/configuration.mdx: tree + hook config example - docs/public/development.mdx: build output line - docs/public/hooks-architecture.mdx: full rewrite of pre-hook section, timing table, performance table - docs/public/architecture/{overview,hooks,worker-service}.mdx: tree comments, JSON config example, Bun requirement section docs/reports/* untouched (historical incident reports). Co-Authored-By: Claude Opus 4.7 (1M context) * fix(install): mergeSettings writes via USER_SETTINGS_PATH Greptile P1 (#2156): `settingsFilePath()` only resolved `process.env.CLAUDE_MEM_DATA_DIR`, while `getSetting()` reads via `USER_SETTINGS_PATH` which `resolveDataDir()` populates from BOTH the env var AND a `CLAUDE_MEM_DATA_DIR` entry persisted in `~/.claude-mem/settings.json`. Result: a user with the data dir saved in settings.json but not exported in their shell would have provider/model settings silently written to `~/.claude-mem/settings.json` while `getSetting()` read from `/custom/path/settings.json` — read/write split. Drop `settingsFilePath()` and the now-unused `homedir` import; reuse the already-imported `USER_SETTINGS_PATH` constant. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(cli): parse --provider, --model, --no-auto-start install flags Greptile P1 (#2156): InstallOptions has fields `provider`, `model`, `noAutoStart`, but the install case in the npx-cli switch only parsed `--ide`. The other three flags were silently dropped — `npx claude-mem install --provider gemini` was a no-op. Extract a `parseInstallOptions(argv)` helper, share it between the bare `npx claude-mem` and `npx claude-mem install` paths, and validate `--provider` against the allowed set. Update help text accordingly. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(install): pipe runtime-setup output, always show IDE multiselect Two issues caught in a docker test of the installer: 1. The bun.sh installer, uv installer, and `bun install` were using stdio: 'inherit', dumping their stdout/stderr through clack's spinner region — visible as raw "downloading uv 0.11.8…" / "Checked 58 installs across 38 packages…" text streaming under the spinner. Switch to stdio: 'pipe' and surface captured stderr only on failure (via a shared describeExecError() helper that includes stdout when stderr is empty). Spinner stays clean on the happy path. 2. promptForIDESelection() silently picked claude-code when no IDEs were detected, never showing the user the multiselect. On a fresh machine with no IDEs present yet (e.g. our docker test container), the user never got to choose. Now: always show the full IDE list when interactive; mark detected ones with [detected] hints and pre-select them; show a warn line if zero are detected explaining they should pick what they plan to use. Non-TTY callers still get the silent claude-code default at the call site (unchanged). Co-Authored-By: Claude Opus 4.7 (1M context) * fix(install): skip marketplace work for claude-code-only, offer to install Claude Code Two related UX fixes from a docker test: **Delay between "Saved Claude model=…" and "Plugin files copied OK"** After dropping the needsManualInstall gate, every install was unconditionally running `copyPluginToMarketplace` (which copied the entire root node_modules tree — thousands of files, dozens of seconds) and `runNpmInstallInMarketplace` (npm install --production) even when only claude-code was selected. Neither is needed for claude-code: that path uses the plugin cache dir + the installed_plugins.json + enabledPlugins flag, all of which we already write. - Drop `node_modules` from `copyPluginToMarketplace`'s allowed-entries list; the dependency-install task populates it on the destination side anyway. - Re-introduce `needsMarketplace = selectedIDEs.some(id => id !== 'claude-code')` scoped *only* to `copyPluginToMarketplace`, `runNpmInstallInMarketplace`, and the pre-install `shutdownWorkerAndWait` (also pointless for claude-code- only flows since we're not overwriting the worker's running cache dir source). All other tasks (cache copy, register, enable, runtime setup) stay unconditional. **Claude Code missing → silent install of an IDE that isn't there** When the user picked claude-code on a machine without it (e.g. a fresh container), the install completed but `claude` was unavailable and the only hint was a generic warn line. Replace with an explicit pre-flight prompt: Claude Code is not installed. Claude-mem works best in Claude Code, but also works with the IDEs below. ? Install Claude Code now? ◆ Yes — install Claude Code (recommended) ◯ No — pick another IDE below ◯ Cancel installation If the user picks "Yes", run `curl -fsSL https://claude.ai/install.sh | bash` (or the PowerShell equivalent on Windows), then re-detect IDEs and proceed with claude-code pre-selected. If the install fails or the user picks "No", the multiselect still appears with claude-code visible (just unmarked [detected]), so they can opt in or pick another IDE. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(install): detect Claude Code via `claude` CLI, not ~/.claude dir The directory `~/.claude` can exist (e.g. mounted in Docker, or created by tooling) without Claude Code actually being installed. Detect the `claude` command in PATH instead so the installer correctly offers to install Claude Code when missing. * docs(learn-codebase): add reviewer note explaining the cost tradeoff The skill intentionally reads every file in full to build a cognitive cache that pays off across the rest of the project. Add a brief note so reviewers (human or bot) understand the tradeoff before flagging the unbounded read as a cost issue. * fix: address Greptile P1 feedback on welcome hint and learn-codebase - SearchRoutes: skip welcome hint when caller passes ?full=true so explicit full-context requests aren't intercepted by the hint. - learn-codebase: replace `sed` instruction with the Read tool's offset/limit parameters, since Bash is gated in Claude Code by default. * feat(install): ASCII-animated logo splash on interactive install Plays a ~1s bloom animation of the claude-mem sunburst logomark when the installer starts in an interactive terminal — geometrically rendered via 12 ray curves around a center disc, in the brand orange. The wordmark and tagline type on alongside the final frame. Auto-skipped on non-TTY, in CI, when NO_COLOR or CLAUDE_MEM_NO_BANNER is set, or when the terminal is too narrow. Inspired by ghostty +boo. * feat(banner): replace rotation frames with angular-sector bloom generator Co-Authored-By: Claude Opus 4.7 (1M context) * feat(banner): replace rotation frames with angular-sector bloom generator Co-Authored-By: Claude Opus 4.7 (1M context) * feat(banner): three-act choreography renderer with radial gradient and diff redraw Co-Authored-By: Claude Opus 4.7 (1M context) * feat(banner): update preview script to support small/medium/hero tier selection Co-Authored-By: Claude Opus 4.7 (1M context) * fix(docker): add COLORTERM=truecolor to test-installer sandbox Co-Authored-By: Claude Opus 4.7 (1M context) * feat(install): auto-apply PATH for Claude Code with spinner UX The Claude Code install.sh prints a Setup notes block telling users to manually edit "your shell config file" to add ~/.local/bin to PATH — which left fresh installs unable to launch claude from the command line. After a successful install, detect ~/.local/bin/claude on disk and, if the dir is missing from PATH, append the right export line to .zshrc / .bash_profile / .bashrc / fish config (idempotent, marked with a comment). Also updates process.env.PATH for the current install run. Wraps the curl|bash install in a clack spinner (interactive only) so the ~4 minute native-build download doesn't look frozen — output is captured silently and dumped on failure for debuggability. Non-interactive mode keeps inherited stdio for CI logs. Verified end-to-end in the test-installer docker sandbox: spinner animates, .bashrc gets the export, fresh login shell resolves claude. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(banner): video-frame ASCII renderer with three-act choreography Generator switched from a single Jimp-rendered logo to pre-extracted video frames concatenated with \x01 separators and gzip-deflated, ported from ghostty's boo wire format. Renderer rewritten around three acts (ignite → stagger bloom → text reveal + breathe) with adaptive sizing, radial gradient, and diff-based redraw. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(onboarding): unify install / SessionStart / viewer around one first-success moment Three surfaces now point at the same north-star moment — open the viewer, do anything in Claude Code, watch an observation appear within seconds — with the same verbatim timing and privacy lines, and a single canonical "how it works" explainer instead of three diverging copies. - Canonical explainer at src/services/worker/onboarding-explainer.md served via GET /api/onboarding/explainer; mirrored into plugin/skills/how-it-works/SKILL.md - SessionStart welcome hint rewritten as third-person status (no imperatives Claude tries to execute), pinned with a default-value regression test - Post-install Next Steps reframed as "two paths": passive default + optional /learn-codebase front-load; drops /mem-search and /knowledge-agent from this surface; adds verbatim timing + privacy lines and /how-it-works link - /api/stats response gains firstObservationAt for the viewer stat row - Viewer WelcomeCard branches on observationCount === 0: empty state shows live worker-connection dot + "waiting for activity"; has-data state shows observations · projects · since [date] and two example prompts. v2 dismiss key - jimp added to package.json to fix pre-existing banner-frame build break Co-Authored-By: Claude Opus 4.7 (1M context) * fix(banner): play unconditionally; only honor CLAUDE_MEM_NO_BANNER The 128-col / TTY / CI / NO_COLOR gates silently swallowed the banner in narrower terminals, CI logs, and any non-TTY pipe — including Docker runs where -it should preserve the experience but column width was the wrong gate. Remove the implicit gates; keep the explicit opt-out only. If a frame wraps in a narrow terminal, that's better than the banner not playing at all. Co-Authored-By: Claude Opus 4.7 (1M context) * revert(banner): restore 15:33 gating logic per user request Reverts eb6fc157. Restores isBannerEnabled to the state at commit 8e448015 (2026-04-30 15:33): TTY check, !CI, !NO_COLOR, !CLAUDE_MEM_NO_BANNER, and cols >= BANNER.width. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(install): wrap remaining slow steps with spinners Each IDE installer (Cursor, Gemini CLI, OpenCode, Windsurf, OpenClaw, Codex CLI, MCP integrations) now runs inside a clack task spinner with per-step progress messages instead of silent dynamic-import + cpSync. Pre-overwrite worker shutdown (up to 10s) and the post-install health probe (up to 3s) also get spinners. Internal console.log/error/warn from each IDE installer is buffered during the spinner; if the install fails, captured output is replayed afterward via log.warn so users can see what broke. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(review): observation count + IDE pre-selection regressions WelcomeCard's "no observations yet" empty state was triggered when a project filter narrowed the feed to zero rows, even with thousands of observations elsewhere. Source the count from global stats.database to match firstObservationAt's scope. Restore initialValues: [] in the IDE multiselect — pre-selecting every detected IDE was the exact regression #2106 was filed for. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(install): trichotomy worker state + cache fallback for script path ensureWorkerStarted now returns 'ready' | 'warming' | 'dead' instead of boolean. The spawned-but-still-warming case (common in Docker cold starts and slow first-time inits) was being misreported as 'did not start', which contradicted the next-steps panel saying 'still starting up'. Install task message and Next Steps headline now agree on the actual state. Also fixes the actual root cause of 'Worker did not start' on claude-code-only installs: the worker script path was hardcoded to the marketplace dir, which is left empty when no non-claude-code IDE is selected. Now falls back to pluginCacheDirectory(version) when the marketplace copy isn't present. Verified end-to-end in docker/claude-mem with --ide claude-code, --ide cursor, and a fresh container — install task and headline agree on 'Worker ready at http://localhost:' in all cases. Co-Authored-By: Claude Opus 4.7 (1M context) * docs: align CLAUDE.md and public docs with current code Sweep across CLAUDE.md and 10 high-traffic docs/public/ MDX files to remove point-in-time references and align with the actual current shape of the codebase. Highlights: - Hardcoded port 37777 → per-user formula (37700 + uid % 100) on the front-door pages (introduction, installation, configuration, architecture/overview, architecture/worker-service, troubleshooting, hooks-architecture, platform-integration). - Default model 'sonnet' → 'claude-haiku-4-5-20251001' (matches SettingsDefaultsManager). - Node 18 → 20 (matches package.json engines). - Lifecycle hook count corrected (5 events). - Removed the nonexistent 'Smart Install' component and pre-built directory tree referencing files that no longer exist (context-hook.ts, save-hook.ts, cleanup-hook.ts, etc.); replaced with the real worker dispatcher shape. - Removed CLAUDE.md '#2101' issue tag (kept the design rationale). - Replaced obsolete hooks.json example with a description of the real bun-runner.js / worker-service.cjs hook event shape. Lower-traffic doc pages still hardcode 37777 — left for a separate global pass. Co-Authored-By: Claude Opus 4.7 (1M context) * chore(scripts): land strip-comments around real parsers (postcss, remark, parse5) Each language gets a real parser to locate comments, then we splice ranges out of the original source. The library never serializes — that's how remark-stringify produced 243 reformat-noise diffs in the first attempt versus the 21 real strip targets here. JS/TS/JSX -> ts.createSourceFile + getLeadingCommentRanges CSS/SCSS -> postcss.parse + walkComments + node.source offsets MD/MDX -> remark-parse (+ remark-mdx) + AST html / mdx-expression nodes HTML -> parse5 with sourceCodeLocationInfo shell/py -> kept hand-rolled hash stripper (no library worth the dep) Preserves: shebangs, @ts-* directives, eslint-disable, biome-ignore, prettier-ignore, triple-slash refs, webpack magic, /*! license keep, @strip-comments-keep file marker. JS/TS handler runs a parse-roundtrip check and refuses to write if syntax errors increased (catches the worker-utils.ts breakage class from the 2026-04-29 attempt). npm scripts: strip-comments (apply) strip-comments:check (CI-style, exits non-zero if changes needed) strip-comments:dry-run (list, no writes) Verified --check on this repo: 21 changes, -4.0% bytes, no parse-error regressions, no reformat-suspect false positives. Co-Authored-By: Claude Opus 4.7 (1M context) * refactor: strip comments codebase-wide via parser-backed tool 21 files changed, -17,550 bytes (-4.0%) of narrative comments removed across .ts / .tsx / .js / .mjs and the .gitignore. JS/TS comments stripped via ts.createSourceFile + getLeadingCommentRanges — same canonical lexer, same behavior as the 2026-04-29 strip, no reformat noise. Preexisting baseline (unchanged): typecheck: 16 errors at HEAD, 16 errors after strip (line numbers shift, no new error classes — verified via diff of sorted error lists) build: fails at HEAD with CrushHooksInstaller.js unresolved import (preexisting, unrelated to this strip) Co-Authored-By: Claude Opus 4.7 (1M context) * fix(install): drop Crush integration references after extract The Crush integration was extracted to its own branch on May 1, but the import at install.ts:280 (and the case block + ide-detection entry + McpIntegrations config + npx-cli help text) still referenced the now- removed CrushHooksInstaller.js, breaking the build. Removes: - case 'crush' block in install.ts - crush entry in ide-detection.ts - CRUSH_CONFIG and registration in McpIntegrations.ts - 'crush' from the IDE Identifiers help line in index.ts Rebuilds worker-service.cjs to match. Co-Authored-By: Claude Opus 4.7 (1M context) * chore(banner): mark generated banner-frames.ts with @strip-comments-keep Without this, every build/strip cycle ping-pongs five lines of doc comments in and out of the auto-generated output. The keep-marker tells strip-comments.ts to skip the file entirely. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(build): drop banner-frame regen from build script generate-banner-frames.mjs requires PNG frames in /tmp/cmem-banner-frames that only exist after the maintainer runs ffmpeg locally on the source video. CI has neither the video nor the frames, so the build broke on Windows. The output (src/npx-cli/banner-frames.ts) is committed, so the regen is a one-shot dev step — not a build step. Run the script directly when the video changes. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(worker): unstick the spinner — kill claim-self-lock, wake on fail, auto-broadcast Three surgical changes that cure the stuck-spinner bug at the source. Phase 1.1 (L9): claimNextMessage no longer self-excludes its own worker_pid. A single UPDATE-RETURNING grabs the oldest pending row by id. Removes the LiveWorkerPidsProvider plumbing that was never injected — Supervisor enforces single-worker via PID file, so the multi-worker SQL was defending against a configuration the project does not support. Phase 1.2 (L19): SessionManager.markMessageFailed wraps PendingMessageStore.markFailed and emits 'message' on the per-session EventEmitter. The iterator's waitForMessage now wakes immediately on re-pend instead of parking for 3 minutes. ResponseProcessor and SessionRoutes routed through the new wrapper. Phase 1.3 (L24): PendingMessageStore takes an optional onMutate callback fired from every mutator (enqueue, claimNextMessage, confirmProcessed, markFailed, transitionMessagesTo, clearFailedOlderThan). SessionManager wires it; WorkerService passes broadcastProcessingStatus. Ten manual broadcast calls deleted across SessionCleanupHelper, SessionEventBroadcaster, SessionRoutes, DataRoutes, and worker-service. Caller discipline becomes structurally impossible to forget. Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(worker): delete dead code — legacy routes, processPendingQueues, decorative guards Pure deletions. Phase 2 of kill-the-asshole-gates. - Legacy /sessions/:sessionDbId/* routes (handleSessionInit, handleObservations, handleSummarize, handleSessionStatus, handleSessionDelete, handleSessionComplete) bypassed all five ingest gates and were a parallel write path. Folded the initializeSession + broadcastNewPrompt + syncUserPrompt + ensureGeneratorRunning + broadcastSessionStarted work into the canonical /api/sessions/init handler so the hook makes one round trip instead of two. - processPendingQueues (~104 lines, zero callers) — replaced in Phase 6 by a one-statement startup sweep. - spawnInProgress Map and crashRecoveryScheduled Set — decorative dedupe over generatorPromise and stillExists checks that already provide the real safety. - STALE_GENERATOR_THRESHOLD_MS — pre-empted live generators and raced with the finally block; the 3min idle timeout already kills zombies. - MAX_SESSION_WALL_CLOCK_MS — ran a SELECT on every observation to enforce 24h. Runaway-spend protection lives in the API key, not in claude-mem. - Missing-id 400 in shared.ts ingestObservation — Zod already enforces min(1) on contentSessionId and toolName at the route schema. - SessionCompletionHandler import + completionHandler field on SessionRoutes (orphaned after handler deletions). Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(worker): SQL-backed getTotalQueueDepth — single source of truth Was: iterate this.sessions.values() and sum getPendingCount per session. Now: SELECT COUNT(*) FROM pending_messages WHERE status IN ('pending','processing'). The in-memory sessions Map drifted from the DB rows whenever a generator exited without confirm/fail, leading to false-positive isProcessing in the UI. Phase 1.3's auto-broadcast fires on every mutation, but it broadcast a stale Map count. Reading from the DB makes the UI's spinner state match what the queue actually holds. Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(worker): typed abortReason replaces wasAborted boolean Was: a boolean wasAborted that lumped every abort together. The finally block branched on !wasAborted, so any abort skipped restart — including idle aborts with pending work, which is exactly the case where we DO want to restart. Now: ActiveSession.abortReason is a typed enum 'idle' | 'shutdown' | 'overflow' | 'restart-guard'. The finally block consumes the reason and only skips restart for 'shutdown' and 'restart-guard'. Idle and overflow aborts fall through, so if pending work exists they trigger restart correctly. Dropped 'stale' and 'wall-clock' from the union — Phase 2 deleted those paths. Natural-completion abort (post-success) intentionally has no reason; it's not gating restart logic. Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(worker): unify the two generator-exit finally blocks Was: worker-service.ts:startSessionProcessor and SessionRoutes:ensureGeneratorRunning each had their own ~70-line finally block with divergent restart-guard handling. The worker-service path called terminateSession on RestartGuard trip and orphaned pending rows (the L16 bug); the SessionRoutes path drained them. Two places to update when rules changed. Now: handleGeneratorExit in src/services/worker/session/GeneratorExitHandler.ts owns the contract: 1. Always kill the SDK subprocess if alive. 2. Always drain processingMessageIds via sessionManager.markMessageFailed (which wakes the iterator — Phase 1.2). 3. shutdown / restart-guard reasons: drain pending rows via transitionMessagesTo('failed'), finalize, remove from Map. Fixes L16. 4. pendingCount=0: finalize normally and remove from Map. 5. pendingCount>0: backoff respawn via per-session respawnTimer (no global Set; Phase 2.4 deleted that). RestartGuard trip drains to 'abandoned'. Both finally blocks are now ~10-line wrappers that translate local state into the canonical abortReason and delegate. Restored completionHandler injection into SessionRoutes (was dropped in Phase 2 cleanup; needed by the unified helper for finalizeSession). Behavior change: SessionRoutes' previous "keep idle session in memory" was deliberately replaced by the plan's "remove from Map on natural completion" — next observation reinitializes via getMessageIterator → initializeSession. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(worker): startup orphan sweep — reset 'processing' rows at boot When the worker dies (crash, kill, restart), any pending_messages rows it left in 'processing' state are by definition orphans (the only worker is dead). Single SQL UPDATE at boot resets them to 'pending' so the iterator can claim them again. Replaces the deleted processPendingQueues function (Phase 2.2). Runs in initializeBackground after dbManager.initialize() and before the initializationComplete middleware releases blocked HTTP requests, so no in-flight request can race the sweep. NOT on a periodic timer — after boot, every 'processing' row has a live consumer and a periodic sweep would race. Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(worker): simplify enqueue catch, replace memorySessionId throw with re-pend 7.1: queueObservation's catch was logging two ERROR-level messages and rethrowing. The rethrow is correct (FK violations / disk full / schema drift should crash loudly), but the verbose ERROR logging pretended the error was recoverable. Reduced to one INFO line + rethrow. 7.2: ResponseProcessor's memorySessionId guard was throwing if the SDK hadn't included session_id on the first user-yield, terminal-failing the entire batch. Now warns and re-pends in-flight messages via sessionManager.markMessageFailed (which wakes the iterator — Phase 1.2). The next iteration tries again with memorySessionId hopefully captured. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(sync): mirror builds to installed-version cache for hot reload When package.json bumps past Claude Code's installed pin, sync-marketplace wrote new code to cache// but the worker loaded from cache//, so worker:restart reloaded the same old code. Replace the exit-on-mismatch preflight with a mirror step: when versions differ, also rsync plugin/ into cache// so worker:restart hot-reloads new code without a Claude Code session restart. The build-version cache still gets written for the eventual `claude plugin update`. Co-Authored-By: Claude Opus 4.7 (1M context) * chore: delete dead barrel files and orphan utilities - src/sdk/index.ts (re-exports parser+prompts; nothing imported the barrel) - src/services/Context.ts (re-exports ./context/index.js; no importers) - src/services/integrations/index.ts (no importers) - src/services/worker/Search.ts (3-line barrel of ./search/index.js) - src/services/infrastructure/index.ts: drop CleanupV12_4_3 re-export - src/utils/error-messages.ts (getWorkerRestartInstructions never imported) - src/types/transcript.ts (170 LoC of types, zero importers) - src/npx-cli/_preview.ts (banner dev preview, no script wires it) Build + tests still pass; observations still flowing. Co-Authored-By: Claude Opus 4.7 (1M context) * chore(parser): drop unused detectLanguage Only the user-grammar-aware variant detectLanguageWithUserGrammars() is actually called. Co-Authored-By: Claude Opus 4.7 (1M context) * chore(types): drop unused SdkSessionRecord + ObservationWithContext Both interfaces in src/types/database.ts had zero importers anywhere in src or tests. Co-Authored-By: Claude Opus 4.7 (1M context) * chore(npx-cli): drop unused getDetectedIDEs + claudeMemDataDirectory getDetectedIDEs has no callers — install.ts uses detectInstalledIDEs directly. claudeMemDataDirectory has no callers either. Co-Authored-By: Claude Opus 4.7 (1M context) * chore(ProcessManager): drop dead orphan-reaper + signal-handler helpers Each had zero callers in src/ or tests/: - cleanupOrphanedProcesses + enumerateOrphanedProcesses - ORPHAN_PROCESS_PATTERNS + ORPHAN_MAX_AGE_MINUTES - forceKillProcess - waitForProcessesExit - createSignalHandler - resetWorkerRuntimePathCache The orphan reaper was retired in PATHFINDER Plan 02 ("OS process groups replace hand-rolled reapers", commit 94d592f2) — these were the leftover pieces. shutdown.ts uses the supervisor's own kill-pgid path instead. parseElapsedTime kept (covered by tests/infrastructure/process-manager.test.ts). Co-Authored-By: Claude Opus 4.7 (1M context) * chore(scripts): delete 11 unreferenced DX/forensic scripts None of these are referenced by package.json npm scripts or docs/. All last touched on Apr 29 only as part of the comment-stripping pass — the feature code itself is older and orphaned: analyze-transformations-smart.js debug-transcript-structure.ts dump-transcript-readable.ts endless-mode-token-calculator.js extract-prompts-to-yaml.cjs extract-rich-context-examples.ts find-silent-failures.sh fix-all-timestamps.ts format-transcript-context.ts test-transcript-parser.ts transcript-to-markdown.ts These are standalone tools — runtime behavior unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) * chore(scripts): delete unused extraction/ and types/ subdirs - scripts/extraction/{extract-all-xml.py, filter-actual-xml.py, README.md} point at ~/Scripts/claude-mem/ — the user's pre-relocation path that no longer exists. Zero references in package.json, src/, or tests/. - scripts/types/export.ts duplicates ObservationRecord etc. and has no importers (CodexCliInstaller imports transcripts/types, not this). Co-Authored-By: Claude Opus 4.7 (1M context) * chore(BranchManager): drop dead getInstalledPluginPath OpenCodeInstaller has its own (used) getInstalledPluginPath; the BranchManager copy never had any external callers. Co-Authored-By: Claude Opus 4.7 (1M context) * chore(ChromaSyncState): unexport DocKind (used internally only) Co-Authored-By: Claude Opus 4.7 (1M context) * test(gemini): drop stale earliestPendingTimestamp / processingMessageIds Both fields were removed from ActiveSession in earlier queue-engine cleanup. Tests had been silently keeping them because the mock sessions use 'as any' to bypass strict typing, so the dead fields rode along without complaint. Co-Authored-By: Claude Opus 4.7 (1M context) * chore: drop 3 unused module-level constants - src/npx-cli/banner.ts: CURSOR_HOME, CLEAR_DOWN (banner uses CLEAR_SCREEN which combines clear-down + cursor-home into a single CSI sequence; the standalone constants were leftovers). - src/services/worker/BranchManager.ts: DEFAULT_SHELL_TIMEOUT_MS (BranchManager only uses GIT_COMMAND_TIMEOUT_MS / NPM_INSTALL_TIMEOUT_MS). Co-Authored-By: Claude Opus 4.7 (1M context) * chore(opencode-plugin): drop dead workerPost helper Only the fire-and-forget variant (workerPostFireAndForget) is actually called. workerPost was the await-result version with no remaining caller. Co-Authored-By: Claude Opus 4.7 (1M context) * chore: drop 8 truly-unused interface fields Verified each by grepping for `.field`, `"field"`, `'field'`, and `field:` patterns across src/ + tests/ + plugin/scripts. Where the only remaining usage was the assignment site, removed the assignments too. - GitHubStarsData: watchers_count, forks_count (only stargazers_count read) - TableColumnInfo: dflt_value (PRAGMA returns it but no caller reads it) - IndexInfo: seq (PRAGMA returns it but no caller reads it) - ObservationRecord: source_files (legacy field, no readers) - HookResult.hookSpecificOutput: permissionDecisionReason - WatchTarget: rescanIntervalMs (set in config, never read) - ShutdownResult: confirmedStopped (write-only — assigned but no reader; updated all 3 return sites to drop it) - ModePrompts: language_instruction (multilingual support never wired) Co-Authored-By: Claude Opus 4.7 (1M context) * chore(npx-cli): reuse InstallOptions type instead of inline duplicate parseInstallOptions had its return type written out inline as an anonymous duplicate of InstallOptions. Use the canonical type (import type — zero bundle cost). Co-Authored-By: Claude Opus 4.7 (1M context) * chore(integrations): drop unused Platform type alias The detectPlatform() function that returned this type was deleted earlier in the branch (along with getScriptExtension that consumed it). The type itself outlived its consumer; only string literals "Platform:" survive in console.log diagnostics, which don't reference the alias. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(worker): broadcast processing_status when summarize is queued broadcastSummarizeQueued was an empty no-op even though handleSummarizeByClaudeId calls it after enqueueing. The PendingMessageStore onMutate callback already fires broadcastProcessingStatus on enqueue, but calling it explicitly from broadcastSummarizeQueued ensures the spinner ticks on the moment a summary is requested even if the onMutate chain has any timing race. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(worker): keep spinner on while summary generates ClaudeProvider's SDK can pull multiple synthetic prompts (e.g. observation + summarize) before producing responses. Each pull pushed an ID to session.processingMessageIds. When the SDK's first observation response came back, ResponseProcessor.confirmProcessed deleted ALL pending message rows — including the still-in-flight summary — so getTotalQueueDepth dropped to 0 and the spinner turned off, even though the summary took another ~22s to actually generate. Tag each in-flight message with its type ({id, type}) so the response processor can pop only the FIFO message of the matching type (observation vs summarize). The summary row stays in 'processing' until its own response arrives, keeping the spinner lit through the entire summary window. Also updates Gemini/OpenRouter providers and GeneratorExitHandler for the new shape. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(worker): clear summary from queue on any SDK response Switch ResponseProcessor from type-aware FIFO matching to strict FIFO popping (each SDK response → 1 in-flight message consumed). This way the summary always clears when the SDK responds, even when the response is unparseable or the summary doesn't actually generate content — preventing stuck spinner / queue-depth-stuck-at-1. Spinner behavior is preserved: messages enqueued after the summary keep the queue depth elevated, and only when the SDK has responded to every prompt does the queue drain to zero. Also: when the consumed message is a 'summarize' and parsing fails, treat it as best-effort and confirmProcessed (no retry) — summaries that can't be parsed shouldn't keep retrying. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(viewer): redesign welcome card and remove source filters The first-start welcome card now explains the three feed card types (observation/summary/prompt) with color-coded badges, points users at the gear icon for settings and the project dropdown for filtering, and plugs /mem-search for recall — replacing the old two-line "ask:" prompts. Source filter tabs (Claude/Codex/etc.) are removed from the header. Filtering by AI provider was nonsense from a user POV; the project dropdown is the only header filter now. Source tracking is also stripped from useSSE, usePagination, App state, and CSS. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(viewer): keep welcome card in feed column, swap rows for 3 squares Two visible problems in the previous design: the card stretched edge-to-edge while feed cards sit in a centered 650px column, and the body was a stack of long horizontal rows that scanned line-by-line. Both fixed: Feed now accepts a pinnedTop slot so the welcome card renders inside the same .feed-content column as observation cards. Body is now a 3-column grid of square feature blocks — Live feed, Tune it, Recall it — each with a custom inline SVG illustration (stacked cards with color-coded stripes, gear+sliders, magnifier over cards). Old text-row sections (welcome-card-types, welcome-card-tips, welcome-card-section, welcome-card-tip-icon) are removed. Squares stack to one column under 600px. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(viewer): convert welcome card to glassy modal with stylized logo Card now opens as a centered modal with a frosted/glass backdrop (blur + saturate) so it doubles as a proper help dialog when reopened from the header's question-mark button. Removed the observation count, project count, and "since" date — those don't make sense for a first-launch surface and felt out of place in a help context. Header art swapped from the small webp logomark to the new high-resolution sun/sunburst PNG (claude-mem-logo-stylized.png), shipped as a checked-in asset in src/ui and plugin/ui. Bigger throughout: 28px h2, 16px tagline, 88px illustrations, 26px feature padding, 1:1 aspect-ratio squares. Backdrop click and Esc both close. Mobile collapses the grid to one column and drops the aspect-ratio constraint. Reverted the unused pinnedTop slot on Feed.tsx since the welcome card is now a true overlay rather than an in-feed pinned card. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(viewer): make welcome modal actually glassy Previous version had a 55%-opacity black backdrop that almost fully blocked the underlying UI — the "glass" was just a dark plate. Now the backdrop is fully transparent (no darkening at all), the panel itself drops to 55% bg-card opacity with its existing backdrop-filter blur(28px) saturate(170%), and the feature squares drop to 35% bg-tertiary so they layer as glass-on-glass over the already-blurred panel. The header and feed below now read clearly through the modal's frosted blur. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(viewer): bulletproof square features via padding-bottom + clamp() fluid type Squares were rendering taller than wide because aspect-ratio is treated as a minimum — content can push the box past 1:1. Switched to the classic padding-bottom: 100% trick: percentage padding resolves against the parent's width, so the box is ALWAYS W × W regardless of content. Inner content sits in an absolutely-positioned flex column that can't push the shell taller. Whole modal is now desktop-first and fluid via clamp() — no media-query stair-steps for type, padding, gaps, border-radius, illustration size, or modal width. Single mobile breakpoint at <600px collapses the grid to one column and reverts the padding-bottom trick so each feature can grow to natural content height. Tightened the three feature descriptions so they fit comfortably inside the square at the desktop size. Co-Authored-By: Claude Opus 4.7 (1M context) * style(viewer): 15% black overlay + heavier modal shadow for elevation Backdrop goes from transparent to rgba(0,0,0,0.15) — just enough darkening to push the modal visually forward without burying the underlying UI. Modal shadow stacked: 40px/120px ambient + 16px/48px contact, both deeper, plus the existing inset 1px highlight. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(build): clear pending_messages queue on build-and-sync Rewrites scripts/clear-failed-queue.ts to talk directly to SQLite via bun:sqlite — the previous HTTP endpoints (/api/pending-queue/*) were removed during the queue engine rewrite, so the script was orphaned. Wires `npm run queue:clear` into `build-and-sync` so each rebuild starts with a clean queue. Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(worker): collapse parser to binary valid/invalid + clearPendingForSession model - Parser: { valid: true, observations, summary } | { valid: false } — drops kind/skipped enum dispatch - ResponseProcessor: two branches only (parseable → store + clearPendingForSession; else → no-op) - Drop processingMessageIds + per-message claim/confirm/markFailed lifecycle across 3 providers - PendingMessageStore: 226 → 140 lines; remove markFailed/transitionMessagesTo/confirmProcessed/clearFailedOlderThan/getAllPending/peekPendingTypes... wait keep peekPendingTypes - Schema migration v31+v32: drop retry_count, failed_at_epoch, completed_at_epoch, worker_pid columns - SessionQueueProcessor: delete two 1s recovery sleeps (let iterator end on error) - Server.ts/SettingsRoutes.ts: replace four magic-number setTimeout exit-flush patterns with flushResponseThen helper - GeneratorExitHandler: 183 → 117 lines (drain in-flight loop gone) Net: -181 lines. No more silent data loss via maxRetries=3. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(pr-2255): address review comments batch 1 - install.ts: needsMarketplace true when claude-code selected (P1, was no-op) - install.ts: throw on invalid --model so CLI exits non-zero - install.ts: skip worker health checks + adapt next-step copy when --no-auto-start - install.ts: repair regenerates plugin cache when missing - index.ts: readFlag rejects missing/flag-shaped values - index.ts: route flag-first invocations (e.g. `--provider claude`) to install - banner.ts: fail-open if frame payload decode throws - SearchRoutes.ts: 5s TTL cache for settings reads on hot hook path (P2) - detect-error-handling-antipatterns.ts: trailing-brace strip whitespace-tolerant - investigate-timestamps.ts: compute Dec 2025 epochs at runtime (was Dec 2024) - regenerate-claude-md.ts: include workingDir in fallback walker so root is covered - sync-marketplace.cjs: parseWorkerPort validates 1..65535 before http.request - sync-to-marketplace.sh: resolve SOURCE_DIR from script location, not cwd - Dockerfile.test-installer: bash --login sources .bashrc via .bash_profile - docs/configuration.mdx: drop nonexistent .worker.port file refs, use settings.json - docs/architecture-overview.md: dynamic port + queue model after parser collapse - docs/architecture/worker-service.mdx: dynamic port example + drop port-file claim - docs/platform-integration.mdx: WORKER_BASE_URL pattern, drop hardcoded 37777 - install/public/install.sh: Node 20 floor (was 18) to match docs Co-Authored-By: Claude Opus 4.7 (1M context) * fix(pr-2255): reset claimed messages to pending on early-return paths ResponseProcessor returns early in two cases: - parser invalid (unparseable response) - memorySessionId not yet captured Both paths previously left the just-claimed message in `status='processing'`, which counts toward `getPendingCount`. The generator-exit handler then sees `pendingCount > 0` and respawns the generator, looping until the restart guard trips and `clearPendingForSession` deletes the message — silent data loss. Calling `resetProcessingToPending` on these paths lets the next generator pass re-claim the message and try again, instead of burning the restart budget on no-op respawns. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(pr-2255): swebench fallback row + troubleshooting port path - evals/swebench/run-batch.py: append fallback prediction row when orchestrator future raises, preserving "never drop an instance" guarantee - docs/troubleshooting.mdx: drop nonexistent .worker.port / worker.port file references; use settings.json + /api/health for port discovery Co-Authored-By: Claude Opus 4.7 (1M context) * fix(pr-2255): memoize per-project observation count for welcome-hint hot path handleContextInject runs on every PostToolUse hook (after every Read/Edit). The welcome-hint block ran a COUNT(*) on observations for every call once CLAUDE_MEM_WELCOME_HINT_ENABLED was true. Observation counts are monotonically increasing — once a project has any observations it always will — so cache the positive result in a Set and skip the COUNT(*) on subsequent requests. Combined with the 5s settings TTL added earlier, the steady-state cost on the hook hot path drops to a Set lookup. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(pr-2255): use clearProcessingForSession on AI-success path clearPendingForSession deletes ALL rows for the session. On the success path of processAgentResponse, that's wrong: messages that arrived as 'pending' during the (1-5s) AI response latency get deleted along with the 'processing' row we just consumed. In a hook burst (three quick PostToolUse hooks), B and C land while A is in flight; A's success then nukes B and C — silent data loss. Add a status-scoped clearProcessingForSession to PendingMessageStore + SessionManager, and use it in ResponseProcessor's success path. The unconditional clearPendingForSession remains correct in GeneratorExitHandler for hard-stop / restart-guard-trip paths. Co-Authored-By: Claude Opus 4.7 (1M context) * Revert "fix(pr-2255): use clearProcessingForSession on AI-success path" This reverts commit a08995299c30cbad36bddc3e5bddda7af8604b35. --------- Co-authored-by: Claude Opus 4.7 (1M context) --- .dockerignore | 2 - .gitattributes | 6 - .github/workflows/claude.yml | 11 +- .../workflows/convert-feature-requests.yml | 1 - .gitignore | 12 - .npmignore | 6 - CLAUDE.md | 12 +- Dockerfile.test-installer | 59 + bunfig.toml | 5 - cursor-hooks/.gitignore | 1 - docker/claude-mem/Dockerfile | 39 - docker/claude-mem/build.sh | 5 - docker/claude-mem/entrypoint.sh | 10 - docker/claude-mem/run.sh | 19 - docs/architecture-overview.md | 36 +- docs/context/agent-sdk-v2-examples.ts | 14 - docs/context/hooks-reference-2026-01-07.md | 1 - docs/i18n/README.ar.md | 1 - docs/public/architecture/hooks.mdx | 18 +- docs/public/architecture/overview.mdx | 150 +-- docs/public/architecture/worker-service.mdx | 32 +- docs/public/beta-features.mdx | 4 +- docs/public/configuration.mdx | 122 +- docs/public/development.mdx | 4 +- docs/public/hooks-architecture.mdx | 63 +- docs/public/installation.mdx | 29 +- docs/public/introduction.mdx | 40 +- docs/public/platform-integration.mdx | 50 +- docs/public/troubleshooting.mdx | 26 +- docs/public/usage/folder-context.mdx | 2 - docs/public/usage/getting-started.mdx | 2 +- evals/swebench/Dockerfile.agent | 32 - evals/swebench/build-agent-image.sh | 6 - evals/swebench/eval.sh | 15 - evals/swebench/run-batch.py | 69 +- evals/swebench/run-instance.sh | 40 - evals/swebench/smoke-test.sh | 31 - evals/swebench/summarize.py | 11 - install/public/install.sh | 9 +- install/public/installer.js | 4 - openclaw/Dockerfile.e2e | 23 - openclaw/e2e-verify.sh | 43 - openclaw/install.sh | 252 +--- openclaw/src/index.test.ts | 54 - openclaw/src/index.ts | 134 +-- openclaw/test-e2e.sh | 6 - openclaw/test-install.sh | 253 ---- openclaw/test-sse-consumer.js | 8 - package.json | 14 +- plans/2026-04-29-installer-streamline.md | 531 +++++++++ plans/2026-04-30-onboarding-ux-overhaul.md | 367 ++++++ plugin/hooks/hooks.json | 8 +- plugin/scripts/bun-runner.js | 69 +- plugin/scripts/context-generator.cjs | 106 +- plugin/scripts/mcp-server.cjs | 50 +- plugin/scripts/smart-install.js | 590 ---------- plugin/scripts/statusline-counts.js | 21 - plugin/scripts/version-check.js | 36 + plugin/scripts/worker-service.cjs | 992 ++++++++-------- plugin/skills/how-it-works/SKILL.md | 22 + .../how-it-works/onboarding-explainer.md | 17 + plugin/skills/learn-codebase/SKILL.md | 21 + .../scripts/generate_changelog.js | 3 - plugin/ui/claude-mem-logo-stylized.png | Bin 0 -> 132855 bytes plugin/ui/viewer-bundle.js | 58 +- plugin/ui/viewer.html | 271 +++-- ragtime/ragtime.ts | 59 +- scripts/analyze-transformations-smart.js | 410 ------- .../detect-error-handling-antipatterns.ts | 52 +- scripts/bug-report/cli.ts | 11 - scripts/bug-report/collector.ts | 6 - scripts/bug-report/index.ts | 10 - scripts/build-hooks.js | 92 +- scripts/build-viewer.js | 4 - scripts/build-worker-binary.js | 4 - scripts/check-pending-queue.ts | 19 - scripts/cleanup-duplicates.ts | 36 - scripts/clear-failed-queue.ts | 235 +--- scripts/cwd-remap.ts | 23 - scripts/debug-transcript-structure.ts | 113 -- scripts/discord-release-notify.js | 11 - scripts/dump-transcript-readable.ts | 99 -- scripts/e2e-knowledge-agents.sh | 28 - scripts/endless-mode-token-calculator.js | 273 ----- scripts/export-memories.ts | 14 - scripts/extract-prompts-to-yaml.cjs | 178 --- scripts/extract-rich-context-examples.ts | 177 --- scripts/extraction/README.md | 82 -- scripts/extraction/extract-all-xml.py | 128 -- scripts/extraction/filter-actual-xml.py | 168 --- scripts/find-silent-failures.sh | 38 - scripts/fix-all-timestamps.ts | 174 --- scripts/fix-corrupted-timestamps.ts | 21 +- scripts/format-transcript-context.ts | 240 ---- scripts/generate-banner-frames.mjs | 143 +++ scripts/generate-changelog.js | 15 - scripts/import-memories.ts | 11 - scripts/investigate-timestamps.ts | 24 +- scripts/publish.js | 13 - scripts/regenerate-claude-md.ts | 111 +- scripts/smart-install.js | 325 ----- scripts/strip-comments.ts | 477 ++++++++ scripts/sync-marketplace.cjs | 110 +- scripts/sync-to-marketplace.sh | 17 +- scripts/test-transcript-parser.ts | 167 --- scripts/transcript-to-markdown.ts | 209 ---- scripts/translate-readme/cli.ts | 9 +- scripts/translate-readme/examples.ts | 20 - scripts/translate-readme/index.ts | 35 - scripts/types/export.ts | 96 -- scripts/validate-timestamp-logic.ts | 13 +- scripts/verify-timestamp-fix.ts | 22 +- scripts/wipe-chroma.cjs | 4 - src/bin/cleanup-duplicates.ts | 7 - src/bin/import-xml-observations.ts | 53 - src/cli/adapters/claude-code.ts | 9 - src/cli/adapters/cursor.ts | 12 - src/cli/adapters/errors.ts | 13 - src/cli/adapters/gemini-cli.ts | 52 +- src/cli/adapters/index.ts | 1 - src/cli/adapters/raw.ts | 2 - src/cli/adapters/windsurf.ts | 15 - src/cli/claude-md-commands.ts | 60 - src/cli/handlers/context.ts | 13 - src/cli/handlers/file-context.ts | 50 +- src/cli/handlers/file-edit.ts | 9 - src/cli/handlers/index.ts | 30 +- src/cli/handlers/observation.ts | 14 - src/cli/handlers/session-init.ts | 44 +- src/cli/handlers/summarize.ts | 24 - src/cli/handlers/user-message.ts | 7 - src/cli/hook-command.ts | 40 +- src/cli/stdin-reader.ts | 57 - src/cli/types.ts | 15 +- src/hooks/hook-response.ts | 7 - src/integrations/opencode-plugin/index.ts | 83 +- src/npx-cli/banner-frames.ts | 21 + src/npx-cli/banner.ts | 180 +++ src/npx-cli/commands/ide-detection.ts | 50 +- src/npx-cli/commands/install.ts | 1043 ++++++++++++----- src/npx-cli/commands/runtime.ts | 51 - src/npx-cli/commands/uninstall.ts | 59 +- src/npx-cli/index.ts | 104 +- src/npx-cli/install/setup-runtime.ts | 271 +++++ src/npx-cli/utils/bun-resolver.ts | 25 +- src/npx-cli/utils/paths.ts | 64 - src/sdk/index.ts | 2 - src/sdk/parser.ts | 95 +- src/sdk/prompts.ts | 42 - src/servers/mcp-server.ts | 115 +- src/services/Context.ts | 8 - src/services/context-generator.ts | 15 - src/services/context/ContextBuilder.ts | 38 - src/services/context/ContextConfigLoader.ts | 10 - src/services/context/ObservationCompiler.ts | 59 +- src/services/context/TokenCalculator.ts | 20 - .../context/formatters/AgentFormatter.ts | 57 +- .../context/formatters/HumanFormatter.ts | 52 +- src/services/context/index.ts | 6 - .../context/sections/FooterRenderer.ts | 12 - .../context/sections/HeaderRenderer.ts | 13 - .../context/sections/SummaryRenderer.ts | 12 - .../context/sections/TimelineRenderer.ts | 26 - src/services/context/types.ts | 41 - src/services/domain/ModeManager.ts | 71 +- src/services/domain/types.ts | 73 +- src/services/infrastructure/CleanupV12_4_3.ts | 58 - .../infrastructure/GracefulShutdown.ts | 39 - src/services/infrastructure/HealthMonitor.ts | 71 -- src/services/infrastructure/ProcessManager.ts | 446 ------- .../infrastructure/WorktreeAdoption.ts | 69 -- src/services/infrastructure/index.ts | 4 - src/services/install/shutdown-helper.ts | 26 +- .../integrations/CodexCliInstaller.ts | 87 -- .../integrations/CursorHooksInstaller.ts | 133 +-- .../integrations/GeminiCliHooksInstaller.ts | 142 --- src/services/integrations/McpIntegrations.ts | 94 -- .../integrations/OpenClawInstaller.ts | 119 -- .../integrations/OpenCodeInstaller.ts | 107 -- src/services/integrations/TelegramNotifier.ts | 12 - .../integrations/WindsurfHooksInstaller.ts | 109 +- src/services/integrations/index.ts | 12 - src/services/integrations/types.ts | 4 - src/services/queue/SessionQueueProcessor.ts | 46 +- src/services/server/ErrorHandler.ts | 29 - src/services/server/Middleware.ts | 8 - src/services/server/Server.ts | 120 +- src/services/server/allowed-constants.ts | 1 - src/services/server/flushResponseThen.ts | 16 + src/services/server/index.ts | 3 - src/services/smart-file-read/parser.ts | 106 -- src/services/smart-file-read/search.ts | 53 +- src/services/sqlite/Database.ts | 63 +- src/services/sqlite/Import.ts | 3 - src/services/sqlite/Observations.ts | 4 - src/services/sqlite/PendingMessageStore.ts | 300 +---- src/services/sqlite/Prompts.ts | 6 - src/services/sqlite/SessionSearch.ts | 113 +- src/services/sqlite/SessionStore.ts | 561 +-------- src/services/sqlite/Sessions.ts | 7 - src/services/sqlite/Summaries.ts | 3 - src/services/sqlite/Timeline.ts | 6 - src/services/sqlite/import/bulk.ts | 23 - src/services/sqlite/index.ts | 8 - src/services/sqlite/migrations.ts | 96 -- src/services/sqlite/migrations/runner.ts | 278 +---- src/services/sqlite/observations/files.ts | 15 - src/services/sqlite/observations/get.ts | 22 - src/services/sqlite/observations/recent.ts | 22 +- src/services/sqlite/observations/store.ts | 23 - src/services/sqlite/observations/types.ts | 26 - src/services/sqlite/prompts/get.ts | 28 - src/services/sqlite/prompts/store.ts | 7 - src/services/sqlite/prompts/types.ts | 12 - src/services/sqlite/schema.sql | 6 +- src/services/sqlite/sessions/create.ts | 28 - src/services/sqlite/sessions/get.ts | 18 - src/services/sqlite/sessions/types.ts | 16 - src/services/sqlite/summaries/get.ts | 26 - src/services/sqlite/summaries/recent.ts | 23 - src/services/sqlite/summaries/store.ts | 16 - src/services/sqlite/summaries/types.ts | 24 - src/services/sqlite/timeline/queries.ts | 43 - src/services/sqlite/transactions.ts | 67 -- src/services/sqlite/types.ts | 66 +- src/services/sync/ChromaMcpManager.ts | 123 +- src/services/sync/ChromaSync.ts | 183 +-- src/services/sync/ChromaSyncState.ts | 24 +- src/services/transcripts/processor.ts | 29 - src/services/transcripts/types.ts | 1 - src/services/transcripts/watcher.ts | 40 +- src/services/worker-service.ts | 503 +------- src/services/worker-spawner.ts | 95 +- src/services/worker-types.ts | 93 +- src/services/worker/BranchManager.ts | 72 +- .../worker/{SDKAgent.ts => ClaudeProvider.ts} | 161 +-- src/services/worker/DatabaseManager.ts | 33 - src/services/worker/FormattingService.ts | 46 - .../{GeminiAgent.ts => GeminiProvider.ts} | 131 +-- ...enRouterAgent.ts => OpenRouterProvider.ts} | 111 +- src/services/worker/PaginationHelper.ts | 47 +- src/services/worker/RestartGuard.ts | 52 +- src/services/worker/SSEBroadcaster.ts | 29 +- src/services/worker/Search.ts | 8 - src/services/worker/SearchManager.ts | 200 +--- src/services/worker/SessionManager.ts | 192 +-- src/services/worker/SettingsManager.ts | 16 - src/services/worker/TimelineService.ts | 37 - .../worker/agents/FallbackErrorHandler.ts | 29 - .../worker/agents/ObservationBroadcaster.ts | 29 - .../worker/agents/ResponseProcessor.ts | 154 +-- .../worker/agents/SessionCleanupHelper.ts | 26 - src/services/worker/agents/index.ts | 16 - src/services/worker/agents/types.ts | 51 +- .../worker/events/SessionEventBroadcaster.ts | 40 - src/services/worker/http/BaseRouteHandler.ts | 29 - src/services/worker/http/middleware.ts | 33 +- .../worker/http/middleware/validateBody.ts | 16 - .../worker/http/routes/ChromaRoutes.ts | 15 - .../worker/http/routes/CorpusRoutes.ts | 51 - src/services/worker/http/routes/DataRoutes.ts | 101 +- src/services/worker/http/routes/LogsRoutes.ts | 35 +- .../worker/http/routes/MemoryRoutes.ts | 39 +- .../worker/http/routes/SearchRoutes.ts | 224 ++-- .../worker/http/routes/SessionRoutes.ts | 636 ++-------- .../worker/http/routes/SettingsRoutes.ts | 108 +- .../worker/http/routes/ViewerRoutes.ts | 33 +- src/services/worker/http/shared.ts | 134 --- .../worker/knowledge/CorpusBuilder.ts | 30 - .../worker/knowledge/CorpusRenderer.ts | 23 - src/services/worker/knowledge/CorpusStore.ts | 24 - .../worker/knowledge/KnowledgeAgent.ts | 60 +- src/services/worker/knowledge/index.ts | 7 - src/services/worker/knowledge/types.ts | 9 +- src/services/worker/onboarding-explainer.md | 17 + src/services/worker/search/ResultFormatter.ts | 63 +- .../worker/search/SearchOrchestrator.ts | 66 -- src/services/worker/search/TimelineBuilder.ts | 41 - src/services/worker/search/errors.ts | 7 - .../worker/search/filters/DateFilter.ts | 20 - .../worker/search/filters/ProjectFilter.ts | 18 - .../worker/search/filters/TypeFilter.ts | 20 - src/services/worker/search/index.ts | 10 - .../search/strategies/ChromaSearchStrategy.ts | 43 - .../search/strategies/HybridSearchStrategy.ts | 34 - .../search/strategies/SQLiteSearchStrategy.ts | 22 - .../search/strategies/SearchStrategy.ts | 30 - src/services/worker/search/types.ts | 47 - .../worker/session/GeneratorExitHandler.ts | 123 ++ .../session/SessionCompletionHandler.ts | 50 +- .../validation/PrivacyCheckValidator.ts | 15 - src/shared/EnvManager.ts | 92 -- src/shared/SettingsDefaultsManager.ts | 119 +- src/shared/hook-constants.ts | 12 +- src/shared/hook-settings.ts | 21 - src/shared/path-utils.ts | 54 +- src/shared/paths.ts | 61 +- src/shared/plugin-state.ts | 10 - src/shared/should-track-project.ts | 38 - src/shared/timeline-formatting.ts | 46 - src/shared/transcript-parser.ts | 38 - src/shared/worker-utils.ts | 205 +--- src/supervisor/env-sanitizer.ts | 20 - src/supervisor/health-checker.ts | 6 - src/supervisor/process-registry.ts | 196 +--- src/supervisor/shutdown.ts | 10 - src/types/database.ts | 68 +- src/types/transcript.ts | 174 --- src/ui/claude-mem-logo-stylized.png | Bin 0 -> 132855 bytes src/ui/viewer-template.html | 271 +++-- src/ui/viewer/App.tsx | 56 +- .../components/ContextSettingsModal.tsx | 6 - src/ui/viewer/components/Feed.tsx | 2 - .../viewer/components/GitHubStarsButton.tsx | 1 - src/ui/viewer/components/Header.tsx | 48 +- src/ui/viewer/components/LogsModal.tsx | 27 +- src/ui/viewer/components/ObservationCard.tsx | 11 +- src/ui/viewer/components/ScrollToTop.tsx | 2 +- src/ui/viewer/components/TerminalPreview.tsx | 2 - src/ui/viewer/components/WelcomeCard.tsx | 218 ++++ src/ui/viewer/constants/api.ts | 4 - src/ui/viewer/constants/settings.ts | 15 - src/ui/viewer/constants/timing.ts | 7 - src/ui/viewer/constants/ui.ts | 6 - src/ui/viewer/hooks/useContextPreview.ts | 2 - src/ui/viewer/hooks/useGitHubStars.ts | 2 - src/ui/viewer/hooks/usePagination.ts | 43 +- src/ui/viewer/hooks/useSSE.ts | 50 +- src/ui/viewer/hooks/useSettings.ts | 9 - src/ui/viewer/hooks/useSpinningFavicon.ts | 9 - src/ui/viewer/hooks/useStats.ts | 1 - src/ui/viewer/hooks/useTheme.ts | 3 - src/ui/viewer/types.ts | 13 +- src/ui/viewer/utils/api.ts | 4 - src/ui/viewer/utils/data.ts | 13 - src/ui/viewer/utils/formatNumber.ts | 10 - src/ui/viewer/utils/formatters.ts | 19 - src/utils/agents-md-utils.ts | 5 - src/utils/claude-md-utils.ts | 145 +-- src/utils/context-injection.ts | 27 - src/utils/cursor-utils.ts | 76 -- src/utils/error-messages.ts | 47 - src/utils/json-utils.ts | 15 - src/utils/logger.ts | 111 -- src/utils/project-filter.ts | 39 +- src/utils/project-name.ts | 37 - src/utils/tag-stripping.ts | 75 +- src/utils/worktree.ts | 28 +- tests/bun-runner.test.ts | 22 - ...test.ts => claude-provider-resume.test.ts} | 37 +- .../cli/adapters/claude-code-subagent.test.ts | 12 - .../handlers/summarize-subagent-skip.test.ts | 28 - .../handlers/summarize-tag-stripping.test.ts | 20 - tests/cli/stdin-reader.test.ts | 8 - .../codex-transcript-watcher-windows.test.ts | 15 - tests/codex-workspace-context.test.ts | 28 - tests/context-injection.test.ts | 9 - .../formatters/agent-formatter.test.ts | 10 - tests/context/observation-compiler.test.ts | 11 - tests/context/token-calculator.test.ts | 32 +- tests/cursor-context-update.test.ts | 22 +- tests/cursor-hooks-json-utils.test.ts | 18 - tests/cursor-mcp-config.test.ts | 25 - tests/cursor-registry.test.ts | 17 - tests/fk-constraint-fix.test.ts | 30 - tests/gemini-cli-compat.test.ts | 33 - ..._agent.test.ts => gemini_provider.test.ts} | 62 +- tests/hook-command.test.ts | 11 - tests/hook-constants.test.ts | 12 - tests/hook-lifecycle.test.ts | 46 - tests/hooks/file-context.test.ts | 16 +- tests/infrastructure/cleanup-v12_4_3.test.ts | 18 +- .../infrastructure/graceful-shutdown.test.ts | 18 - tests/infrastructure/health-monitor.test.ts | 20 +- .../plugin-disabled-check.test.ts | 9 - .../plugin-distribution.test.ts | 31 +- tests/infrastructure/process-manager.test.ts | 77 +- .../version-consistency.test.ts | 27 +- tests/infrastructure/wmic-parsing.test.ts | 15 - .../infrastructure/worker-json-status.test.ts | 126 -- tests/install-non-tty.test.ts | 56 +- tests/integration/chroma-vector-sync.test.ts | 60 - tests/integration/hook-execution-e2e.test.ts | 27 - .../integration/worker-api-endpoints.test.ts | 33 +- tests/json-utils.test.ts | 7 - tests/log-level-audit.test.ts | 46 +- tests/logger-usage-standards.test.ts | 34 - tests/mcp-integrations.test.ts | 18 - tests/npx-search-query-param.test.ts | 24 - tests/plugin-scripts-line-endings.test.ts | 15 - tests/sdk/parse-summary.test.ts | 17 - tests/sdk/parser.test.ts | 6 - tests/server/error-handler.test.ts | 13 - tests/server/server.test.ts | 25 - tests/servers/mcp-tool-schemas.test.ts | 11 - tests/services/logs-routes-tail-read.test.ts | 17 +- .../queue/SessionQueueProcessor.test.ts | 67 +- .../sqlite/PendingMessageStore.test.ts | 26 +- .../get-observations-by-ids-relevance.test.ts | 17 +- .../services/sqlite/migration-runner.test.ts | 56 +- .../observations/store-subagent-label.test.ts | 25 - tests/services/sqlite/parse-file-list.test.ts | 6 - tests/services/sqlite/schema-repair.test.ts | 35 - .../session-search-path-matching.test.ts | 12 - .../session-store-mark-completed.test.ts | 6 - .../stale-abort-controller-guard.test.ts | 22 +- .../sync/chroma-mcp-manager-cwd.test.ts | 19 - .../sync/chroma-mcp-manager-ssl.test.ts | 16 - .../services/worker-daemon-port-race.test.ts | 16 - tests/services/worker-spawner.test.ts | 14 - tests/session_id_usage_validation.test.ts | 41 +- tests/session_store.test.ts | 20 +- tests/setup-runtime.test.ts | 113 ++ .../shared/settings-defaults-manager.test.ts | 44 +- tests/shared/timeline-formatting.test.ts | 5 - tests/shared/welcome-hint-default.test.ts | 67 ++ tests/smart-install.test.ts | 356 ------ tests/sqlite/data-integrity.test.ts | 16 - tests/sqlite/observations.test.ts | 47 +- tests/sqlite/prompts.test.ts | 15 - tests/sqlite/sessions.test.ts | 16 - tests/sqlite/summaries.test.ts | 15 +- tests/sqlite/transactions.test.ts | 21 - tests/supervisor/env-sanitizer.test.ts | 6 - tests/supervisor/health-checker.test.ts | 4 - tests/supervisor/index.test.ts | 8 - tests/supervisor/process-registry.test.ts | 8 - tests/supervisor/shutdown.test.ts | 4 - tests/utils/claude-md-utils.test.ts | 65 +- tests/utils/logger-format-tool.test.ts | 28 - tests/utils/project-filter.test.ts | 6 - tests/utils/project-name-isolation.test.ts | 12 - tests/utils/project-name.test.ts | 6 - tests/utils/skill-docs-placement.test.ts | 7 - tests/utils/tag-stripping.test.ts | 21 +- tests/viewer/welcome-card-storage.test.ts | 62 + tests/worker-spawn.test.ts | 29 +- .../SearchManager.timeline-anchor.test.ts | 74 +- .../agents/fallback-error-handler.test.ts | 15 +- .../worker/agents/response-processor.test.ts | 39 - .../agents/session-cleanup-helper.test.ts | 23 - .../routes/corpus-routes-coercion.test.ts | 12 - .../http/routes/data-routes-coercion.test.ts | 21 - .../worker/http/routes/memory-routes.test.ts | 10 - .../routes/search-routes-welcome-hint.test.ts | 156 +++ .../middleware/cors-restriction.test.ts | 17 +- tests/worker/search/result-formatter.test.ts | 14 +- .../worker/search/search-orchestrator.test.ts | 11 - .../strategies/chroma-search-strategy.test.ts | 36 +- .../strategies/hybrid-search-strategy.test.ts | 17 +- .../strategies/sqlite-search-strategy.test.ts | 3 +- tests/zombie-prevention.test.ts | 115 +- 452 files changed, 6189 insertions(+), 21059 deletions(-) create mode 100644 Dockerfile.test-installer create mode 100644 plans/2026-04-29-installer-streamline.md create mode 100644 plans/2026-04-30-onboarding-ux-overhaul.md delete mode 100644 plugin/scripts/smart-install.js create mode 100644 plugin/scripts/version-check.js create mode 100644 plugin/skills/how-it-works/SKILL.md create mode 100644 plugin/skills/how-it-works/onboarding-explainer.md create mode 100644 plugin/skills/learn-codebase/SKILL.md create mode 100644 plugin/ui/claude-mem-logo-stylized.png delete mode 100644 scripts/analyze-transformations-smart.js delete mode 100644 scripts/debug-transcript-structure.ts delete mode 100644 scripts/dump-transcript-readable.ts delete mode 100644 scripts/endless-mode-token-calculator.js delete mode 100644 scripts/extract-prompts-to-yaml.cjs delete mode 100644 scripts/extract-rich-context-examples.ts delete mode 100644 scripts/extraction/README.md delete mode 100755 scripts/extraction/extract-all-xml.py delete mode 100755 scripts/extraction/filter-actual-xml.py delete mode 100755 scripts/find-silent-failures.sh delete mode 100755 scripts/fix-all-timestamps.ts delete mode 100644 scripts/format-transcript-context.ts create mode 100644 scripts/generate-banner-frames.mjs delete mode 100644 scripts/smart-install.js create mode 100644 scripts/strip-comments.ts delete mode 100644 scripts/test-transcript-parser.ts delete mode 100644 scripts/transcript-to-markdown.ts delete mode 100644 scripts/types/export.ts create mode 100644 src/npx-cli/banner-frames.ts create mode 100644 src/npx-cli/banner.ts create mode 100644 src/npx-cli/install/setup-runtime.ts delete mode 100644 src/sdk/index.ts delete mode 100644 src/services/Context.ts delete mode 100644 src/services/integrations/index.ts create mode 100644 src/services/server/flushResponseThen.ts rename src/services/worker/{SDKAgent.ts => ClaudeProvider.ts} (62%) rename src/services/worker/{GeminiAgent.ts => GeminiProvider.ts} (72%) rename src/services/worker/{OpenRouterAgent.ts => OpenRouterProvider.ts} (76%) delete mode 100644 src/services/worker/Search.ts create mode 100644 src/services/worker/onboarding-explainer.md create mode 100644 src/services/worker/session/GeneratorExitHandler.ts delete mode 100644 src/types/transcript.ts create mode 100644 src/ui/claude-mem-logo-stylized.png create mode 100644 src/ui/viewer/components/WelcomeCard.tsx delete mode 100644 src/utils/error-messages.ts rename tests/{sdk-agent-resume.test.ts => claude-provider-resume.test.ts} (70%) delete mode 100644 tests/codex-workspace-context.test.ts rename tests/{gemini_agent.test.ts => gemini_provider.test.ts} (80%) delete mode 100644 tests/npx-search-query-param.test.ts create mode 100644 tests/setup-runtime.test.ts create mode 100644 tests/shared/welcome-hint-default.test.ts delete mode 100644 tests/smart-install.test.ts create mode 100644 tests/viewer/welcome-card-storage.test.ts create mode 100644 tests/worker/http/routes/search-routes-welcome-hint.test.ts diff --git a/.dockerignore b/.dockerignore index 6609c135..17893864 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,5 +1,3 @@ -# Keep the build context small for evals/swebench/Dockerfile.agent. -# The Dockerfile needs `plugin/` and `evals/swebench/` — do NOT exclude them. node_modules/ .git/ logs/ diff --git a/.gitattributes b/.gitattributes index 443ffe6b..75e74b3b 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,14 +1,8 @@ -# Normalize all text files to LF on commit and checkout. -# This prevents CRLF shebang lines in bundled scripts from breaking -# the MCP server on macOS/Linux when built on Windows. Fixes #1342. * text=auto eol=lf -# Compiled plugin scripts must always be LF — CRLF in the shebang -# causes "env: node\r: No such file or directory" on non-Windows hosts. plugin/scripts/*.cjs eol=lf plugin/scripts/*.js eol=lf -# Explicitly mark binary assets so git never modifies them. *.png binary *.jpg binary *.jpeg binary diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml index 687f7cc5..98bfd01f 100644 --- a/.github/workflows/claude.yml +++ b/.github/workflows/claude.yml @@ -23,7 +23,7 @@ jobs: pull-requests: read issues: read id-token: write - actions: read # Required for Claude to read CI results on PRs + actions: read steps: - name: Checkout repository uses: actions/checkout@v6 @@ -36,15 +36,6 @@ jobs: with: claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} - # This is an optional setting that allows Claude to read CI results on PRs additional_permissions: | actions: read - # Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it. - # prompt: 'Update the pull request description to include a summary of changes.' - - # Optional: Add claude_args to customize behavior and configuration - # See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md - # or https://docs.claude.com/en/docs/claude-code/cli-reference for available options - # claude_args: '--allowed-tools Bash(gh pr:*)' - diff --git a/.github/workflows/convert-feature-requests.yml b/.github/workflows/convert-feature-requests.yml index f23b78f6..0ad1a25d 100644 --- a/.github/workflows/convert-feature-requests.yml +++ b/.github/workflows/convert-feature-requests.yml @@ -13,7 +13,6 @@ on: jobs: convert: runs-on: ubuntu-latest - # Only run on labeled event if the label is 'feature-request', or always run on workflow_dispatch if: | (github.event_name == 'issues' && github.event.label.name == 'feature-request') || github.event_name == 'workflow_dispatch' diff --git a/.gitignore b/.gitignore index 07ecdb07..511b2961 100644 --- a/.gitignore +++ b/.gitignore @@ -21,14 +21,11 @@ bun.lock private/ Auto Run Docs/ -# Generated UI files (built from viewer-template.html) src/ui/viewer.html -# Local MCP server config (for development only) .mcp.json .cursor/ -# Ignore WebStorm project files (for dinosaur IDE users) .idea/ .claude-octopus/ @@ -37,23 +34,14 @@ src/ui/viewer.html .claude/scheduled_tasks.lock .octo/ -# Installer marker — dropped by the claude-mem CLI at install time plugin/.cli-installed -# Compiled macOS binary — produced on demand by `npm run build:cli-binary`, -# never committed. Committed binaries drift from source and end up shipping -# since-removed routes/handlers and stale version constants (see #2158, -# #2200, #2154). The JS fallback (bun-runner.js → worker-service.cjs) covers -# all functionality on every platform. plugin/scripts/claude-mem -# Local contribution analysis (not part of upstream) CONTRIB_NOTES.md -# Docker container runtime data (basic claude-mem container) .docker-claude-mem-data/ -# SWE-bench eval outputs evals/swebench/runs/ claude-opus-4-7+claude-mem.*.json logs/run_evaluation/ diff --git a/.npmignore b/.npmignore index 1aab776a..8df7aa66 100644 --- a/.npmignore +++ b/.npmignore @@ -1,4 +1,3 @@ -# Source code (dist/ and plugin/ are the shipped artifacts) src/ scripts/ tests/ @@ -7,14 +6,12 @@ datasets/ private/ antipattern-czar/ -# Heavy binaries installed at runtime via smart-install.js plugin/node_modules/ plugin/scripts/claude-mem plugin/bun.lock plugin/data/ plugin/data.backup/ -# Development files *.ts !*.d.ts tsconfig*.json @@ -24,7 +21,6 @@ tsconfig*.json jest.config* vitest.config* -# Git and CI .git/ .github/ .gitignore @@ -33,14 +29,12 @@ vitest.config* .mcp.json .plan/ -# OS files .DS_Store *.log *.tmp *.temp Thumbs.db -# Misc Auto Run Docs/ ~*/ http*/ diff --git a/CLAUDE.md b/CLAUDE.md index 9f489782..9b556358 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,9 +6,9 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions. **5 Lifecycle Hooks**: SessionStart → UserPromptSubmit → PostToolUse → Summary → SessionEnd -**Hooks** (`src/hooks/*.ts`) - TypeScript → ESM, built to `plugin/scripts/*-hook.js` +**Hooks** (`src/hooks/*.ts`) - TypeScript hook logic compiled into the unified worker dispatcher (`plugin/scripts/worker-service.cjs`). Lifecycle hook entries in `plugin/hooks/hooks.json` invoke the worker via `bun-runner.js`. The Setup-phase `version-check.js` is the only standalone hook script. -**Worker Service** (`src/services/worker-service.ts`) - Express API on port 37777, Bun-managed, handles AI processing asynchronously +**Worker Service** (`src/services/worker-service.ts`) - Express API on the per-user worker port (default `37700 + (uid % 100)`, configurable via `CLAUDE_MEM_WORKER_PORT`), Bun-managed, handles AI processing asynchronously **Database** (`src/services/sqlite/`) - SQLite3 at `~/.claude-mem/claude-mem.db` @@ -20,7 +20,7 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions. **Chroma** (`src/services/sync/ChromaSync.ts`) - Vector embeddings for semantic search -**Viewer UI** (`src/ui/viewer/`) - React interface at http://localhost:37777, built to `plugin/ui/viewer.html` +**Viewer UI** (`src/ui/viewer/`) - React interface served by the worker on its configured port (default `http://127.0.0.1:`), built to `plugin/ui/viewer.html` ## Privacy Tags - `content` - User-level privacy control (manual, prevents storage) @@ -55,7 +55,7 @@ Claude-mem supports running multiple isolated profiles on the same machine (e.g. - **All paths and ports derive from these two env vars.** Hooks, npx-cli (`install`/`uninstall`/`start`/`search`), the OpenCode plugin, the OpenClaw installer, and the timeline-report skill all honor them. The settings file itself lives at `$CLAUDE_MEM_DATA_DIR/settings.json`. -- **Closes #2101.** See `src/shared/SettingsDefaultsManager.ts` for the canonical port/data-dir defaults and `plugin/skills/timeline-report/SKILL.md` for the shell snippet that resolves the port for arbitrary skills. +- See `src/shared/SettingsDefaultsManager.ts` for the canonical port/data-dir defaults and `plugin/skills/timeline-report/SKILL.md` for the shell snippet that resolves the port for arbitrary skills. ## File Locations @@ -95,14 +95,14 @@ Claude-mem is designed with a clean separation between open-source core function **Open-Source Core** (this repository): -- All worker API endpoints on localhost:37777 remain fully open and accessible +- All local worker HTTP API endpoints (per-user port — see Architecture above) remain fully open and accessible - Pro features are headless - no proprietary UI elements in this codebase - Pro integration points are minimal: settings for license keys, tunnel provisioning logic - The architecture ensures Pro features extend rather than replace core functionality **Pro Features** (coming soon, external): -- Enhanced UI (Memory Stream) connects to the same localhost:37777 endpoints as the open viewer +- Enhanced UI (Memory Stream) connects to the same local worker endpoints as the open viewer - Additional features like advanced filtering, timeline scrubbing, and search tools - Access gated by license validation, not by modifying core endpoints - Users without Pro licenses continue using the full open-source viewer UI without limitation diff --git a/Dockerfile.test-installer b/Dockerfile.test-installer new file mode 100644 index 00000000..0ac91bce --- /dev/null +++ b/Dockerfile.test-installer @@ -0,0 +1,59 @@ +FROM ubuntu:24.04 +ARG NODE_VERSION=20 + +ENV DEBIAN_FRONTEND=noninteractive +ENV TERM=xterm-256color +ENV COLORTERM=truecolor + +RUN apt-get update && apt-get install -y --no-install-recommends \ + curl \ + ca-certificates \ + bash \ + git \ + build-essential \ + python3 \ + unzip \ + jq \ + sudo \ + && rm -rf /var/lib/apt/lists/* + +RUN curl -fsSL https://deb.nodesource.com/setup_${NODE_VERSION}.x | bash - \ + && apt-get install -y nodejs \ + && rm -rf /var/lib/apt/lists/* + +RUN node -v && npm -v + +WORKDIR /workspace + +RUN cat > /root/.bashrc <<'EOF' +export PS1='\[\033[1;36m\]cmem-test\[\033[0m\]:\[\033[1;33m\]\w\[\033[0m\]\$ ' + +cat <<'BANNER' +===================================================================== + claude-mem installer test sandbox (clean Linux, no Bun, no uv) +===================================================================== + + Try the new installer interactively: + + node dist/npx-cli/index.js install --no-auto-start + + Or just the runtime setup module via repair: + + node dist/npx-cli/index.js repair + + After install, verify the Setup hook is fast: + + time node ~/.claude/plugins/cache/thedotmack/claude-mem/$(jq -r .version package.json)/scripts/version-check.js + + Container HOME=/root is isolated — nothing here touches your real + ~/.claude or ~/.claude-mem. Exit with Ctrl-D. + +===================================================================== +BANNER +EOF + +# bash --login reads ~/.bash_profile (or ~/.profile), not ~/.bashrc, so +# without this the banner above never runs in the container's CMD shell. +RUN printf '%s\n' '[[ -f ~/.bashrc ]] && . ~/.bashrc' > /root/.bash_profile + +CMD ["bash", "--login"] diff --git a/bunfig.toml b/bunfig.toml index dbf0d8f5..047f1478 100644 --- a/bunfig.toml +++ b/bunfig.toml @@ -1,7 +1,2 @@ [test] -# Force each test file into its own worker process. -# Prevents mock.module() calls (which are permanent within a worker) -# from leaking across test files in parallel runs. -# Note: smol=true increases test startup time by spawning one Bun process per file. -# See: https://github.com/thedotmack/claude-mem/issues/1299 smol = true diff --git a/cursor-hooks/.gitignore b/cursor-hooks/.gitignore index add7b814..b05de43f 100644 --- a/cursor-hooks/.gitignore +++ b/cursor-hooks/.gitignore @@ -1,3 +1,2 @@ -# Ignore backup files created by sed *.bak diff --git a/docker/claude-mem/Dockerfile b/docker/claude-mem/Dockerfile index ce896dc9..a2cab0a4 100644 --- a/docker/claude-mem/Dockerfile +++ b/docker/claude-mem/Dockerfile @@ -1,23 +1,3 @@ -# Basic claude-mem container for ad-hoc testing. -# -# Base layout mirrors anthropics/claude-code .devcontainer -# (https://github.com/anthropics/claude-code/blob/main/.devcontainer/Dockerfile): -# FROM node:20, non-root `node` user, global npm install of @anthropic-ai/claude-code. -# We skip the firewall/zsh/fzf/delta/git-hist noise since this image is for -# exercising claude-mem, not as a full dev environment. -# -# On top of that base we install: -# - Bun (claude-mem worker service runtime) -# - uv (provides Python for Chroma per CLAUDE.md) -# - The locally-built plugin/ tree at /opt/claude-mem -# -# Usage: -# docker build -f docker/claude-mem/Dockerfile -t claude-mem:basic . -# docker run --rm -it \ -# -v $(mktemp -d):/home/node/.claude-mem \ -# -e CLAUDE_MEM_CREDENTIALS_FILE=/auth/.credentials.json \ -# -v /path/to/extracted/creds.json:/auth/.credentials.json:ro \ -# claude-mem:basic FROM node:20 @@ -36,56 +16,37 @@ RUN apt-get update \ sqlite3 \ && apt-get clean && rm -rf /var/lib/apt/lists/* -# Bun — system-wide so the unprivileged `node` user can execute it. -# Pin via --build-arg BUN_VERSION=X.Y.Z; default is the version verified at PR time. ARG BUN_VERSION=1.3.12 ENV BUN_INSTALL="/usr/local/bun" RUN curl -fsSL https://bun.sh/install | bash -s "bun-v${BUN_VERSION}" \ && chmod -R a+rX /usr/local/bun ENV PATH="/usr/local/bun/bin:${PATH}" -# uv — system-wide, for Chroma's Python runtime. Pin via --build-arg UV_VERSION=X.Y.Z. -# Versioned installer URL per https://docs.astral.sh/uv/getting-started/installation/. ARG UV_VERSION=0.11.7 ENV UV_INSTALL_DIR="/usr/local/bin" -# `&&` binds tighter than `||` in bash, so the previous form let `curl|sh` fail -# silently via the trailing `|| true`. Group the chmod so tolerated failure is -# scoped to perms-fixing only. RUN set -eux \ && curl -LsSf "https://astral.sh/uv/${UV_VERSION}/install.sh" | sh \ && { chmod a+rX /usr/local/bin/uv /usr/local/bin/uvx 2>/dev/null || true; } -# Match the upstream devcontainer's npm-global prefix so `npm install -g` -# targets a dir the `node` user owns. RUN mkdir -p /usr/local/share/npm-global \ && chown -R node:node /usr/local/share/npm-global ENV NPM_CONFIG_PREFIX=/usr/local/share/npm-global ENV PATH="/usr/local/share/npm-global/bin:${PATH}" -# Claude Code CLI. Override at build-time with --build-arg CLAUDE_CODE_VERSION=X.Y.Z -# to pin; default tracks latest. ARG CLAUDE_CODE_VERSION=latest USER node RUN npm install -g @anthropic-ai/claude-code@${CLAUDE_CODE_VERSION} -# Locally-built claude-mem plugin. COPY runs as root by default and layers are -# cached, so put this after the npm install so iterating on the plugin doesn't -# invalidate the CLI install layer. USER root COPY plugin/ /opt/claude-mem/ RUN chown -R node:node /opt/claude-mem -# Persistent mount points for ad-hoc testing — mount a host dir at either of -# these to inspect the claude-mem DB after a session. RUN mkdir -p /home/node/.claude /home/node/.claude-mem \ && chown -R node:node /home/node/.claude /home/node/.claude-mem USER node WORKDIR /home/node -# Helper: copies OAuth creds out of the read-only mount into $HOME/.claude/ -# before exec'ing whatever you asked for. Saves the "cp + chmod" dance every -# time you drop in. COPY --chown=node:node docker/claude-mem/entrypoint.sh /usr/local/bin/claude-mem-entrypoint RUN chmod +x /usr/local/bin/claude-mem-entrypoint diff --git a/docker/claude-mem/build.sh b/docker/claude-mem/build.sh index a587329c..9c33f6af 100755 --- a/docker/claude-mem/build.sh +++ b/docker/claude-mem/build.sh @@ -1,9 +1,4 @@ #!/usr/bin/env bash -# Build the basic claude-mem Docker image from the current worktree. -# -# Usage: -# docker/claude-mem/build.sh # builds claude-mem:basic -# TAG=my-tag docker/claude-mem/build.sh # override the tag set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/docker/claude-mem/entrypoint.sh b/docker/claude-mem/entrypoint.sh index 033ef921..f6847be0 100755 --- a/docker/claude-mem/entrypoint.sh +++ b/docker/claude-mem/entrypoint.sh @@ -1,12 +1,4 @@ #!/usr/bin/env bash -# Entrypoint for the basic claude-mem container. Seeds OAuth creds if a -# credentials file is mounted, then exec's whatever was passed (default: bash). -# -# Env vars: -# CLAUDE_MEM_CREDENTIALS_FILE Path to a mounted OAuth credentials JSON file -# (e.g. /auth/.credentials.json). Copied into -# $HOME/.claude/.credentials.json at startup. -# ANTHROPIC_API_KEY Standard API-key auth; set when OAuth isn't used. set -euo pipefail @@ -21,8 +13,6 @@ if [[ -n "${CLAUDE_MEM_CREDENTIALS_FILE:-}" ]]; then chmod 600 "$HOME/.claude/.credentials.json" fi -# Helpful one-liner for interactive users: run `claude` with the plugin dir -# preconfigured. Don't force it — `exec "$@"` lets you override freely. export PATH="/usr/local/bun/bin:/usr/local/share/npm-global/bin:$PATH" exec "$@" diff --git a/docker/claude-mem/run.sh b/docker/claude-mem/run.sh index 4d8f62a5..5015f71a 100755 --- a/docker/claude-mem/run.sh +++ b/docker/claude-mem/run.sh @@ -1,13 +1,4 @@ #!/usr/bin/env bash -# Drop into an interactive claude-mem container with OAuth creds + persistent -# memory volume. For ad-hoc testing / poking around. -# -# Usage: -# docker/claude-mem/run.sh -# docker/claude-mem/run.sh claude --plugin-dir /opt/claude-mem --print "hi" -# -# On exit, the mounted .claude-mem/ dir on the host survives so you can inspect -# the DB: `sqlite3 /claude-mem.db 'select count(*) from observations'`. set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -18,17 +9,12 @@ HOST_MEM_DIR="${HOST_MEM_DIR:-$REPO_ROOT/.docker-claude-mem-data}" mkdir -p "$HOST_MEM_DIR" echo "[run] host .claude-mem dir: $HOST_MEM_DIR" >&2 -# Auth. Prefer OAuth (extracted from macOS Keychain / Linux creds file); -# fall back to ANTHROPIC_API_KEY env. CREDS_FILE="" CREDS_MOUNT_ARGS=() if [[ -z "${ANTHROPIC_API_KEY:-}" ]]; then CREDS_FILE="$(mktemp -t claude-mem-creds.XXXXXX.json)" trap 'rm -f "$CREDS_FILE"' EXIT - # Try macOS Keychain first (primary storage on Darwin), then fall back to - # the on-disk credentials file — some macOS setups (older CLI versions, - # users who migrated machines) still have the file-only form. creds_obtained=0 if [[ "$(uname)" == "Darwin" ]]; then if security find-generic-password -s 'Claude Code-credentials' -w > "$CREDS_FILE" 2>/dev/null \ @@ -55,14 +41,9 @@ else CREDS_MOUNT_ARGS=(-e ANTHROPIC_API_KEY) fi -# Pick -it only when a TTY is attached (keeps non-interactive callers working). -# Initialize empty; expansion below safely omits args when the array is unset/empty. TTY_ARGS=() [[ -t 0 && -t 1 ]] && TTY_ARGS=(-it) -# NOT `exec` — we want the EXIT trap above to run and remove $CREDS_FILE -# after the container exits. Running docker as a child keeps the shell -# alive long enough for the trap to fire. docker run --rm ${TTY_ARGS[@]+"${TTY_ARGS[@]}"} \ "${CREDS_MOUNT_ARGS[@]}" \ -v "$HOST_MEM_DIR:/home/node/.claude-mem" \ diff --git a/docs/architecture-overview.md b/docs/architecture-overview.md index 2882780a..14878995 100644 --- a/docs/architecture-overview.md +++ b/docs/architecture-overview.md @@ -14,7 +14,7 @@ | +-- handlers/ (context, session-init, observation, | | summarize, session-complete) | +-----------------------------------------------------------+ -| Worker Daemon (Express, port 37777) | +| Worker Daemon (Express, per-user port 37700+(uid%100)) | | +-- SessionManager (session lifecycle) | | +-- SDKAgent (Claude Agent SDK) | | +-- SearchManager (search orchestration) | @@ -32,13 +32,15 @@ | Event | Handler | What it does | Timeout | |-------|---------|-------------|---------| -| Setup | setup.sh | Install system dependencies | 300s | -| SessionStart | smart-install.js + context | Install deps + start worker + inject context | 60s | +| Setup | version-check.js | Sub-100ms version-marker check; prompts `npx claude-mem repair` on mismatch | 60s | +| SessionStart | worker start + context | Start worker service and inject context | 60s | | UserPromptSubmit | session-init | Register session + start SDK agent + semantic injection | 60s | | PostToolUse | observation | Capture tool usage -> enqueue in worker | 120s | | Summary | summarize | Request session summary from SDK agent | 120s | | SessionEnd | session-complete | End session + drain pending messages | 30s | +On first install, `npx claude-mem install` sets up Bun and uv globally, runs `bun install` in the plugin cache, and writes an `.install-version` marker — all behind a visible clack spinner. The Setup hook then runs `version-check.js` on every Claude Code startup; if the plugin was upgraded externally (e.g. `claude plugin update`), it writes a hint to stderr asking the user to run `npx claude-mem repair`. The hook always exits 0 (non-blocking). + ## Data Flow ```text @@ -62,27 +64,29 @@ Stop -> summarize -> /api/sessions/summarize ## Key Patterns -### CLAIM-CONFIRM (PendingMessageStore) +### Pending Queue (PendingMessageStore) ```text -enqueue() -> INSERT status='pending' -claimNextMessage() -> UPDATE status='processing' (atomic) -confirmProcessed() -> DELETE (success) -markFailed() -> UPDATE status='failed' (retry < 3) - -Self-healing: messages in 'processing' for >60s reset to 'pending' +enqueue() -> INSERT row with `pending` status +clearPendingForSession() -> DELETE all pending rows for session + (called whenever the parser returns + a parseable response, regardless of + whether observations were extracted) ``` -### Circuit-Breaker (SessionRoutes) +Parser is binary: `{ valid: true, observations, summary }` or `{ valid: false }`. +Unparseable responses leave the queue untouched and the session iterator continues. + +### Generator restart loop (SessionRoutes) ```text Generator crash -> retry 1 (1s) -> retry 2 (2s) -> retry 3 (4s) - -> consecutiveRestarts > 3 -> CIRCUIT-BREAKER - -> markAllSessionMessagesAbandoned(sessionDbId) - -> Stop. No infinite loop. + -> consecutiveRestarts > 3 -> stop and let the iterator end ``` -Counter resets to 0 when generator completes work naturally. +Counter resets to 0 when generator completes work naturally. Pending +messages remain in the queue across restarts and are cleared by the +parser path on the next valid response. ### Graceful Degradation (hook-command.ts) @@ -117,7 +121,7 @@ The conversion between them is handled by SessionStore and is critical for FK co | observations | memory_session_id, type, title, narrative, content_hash | Tool usage observations | | session_summaries | memory_session_id, request, learned, completed | Session summaries | | user_prompts | content_session_id, prompt_text | User prompt history | -| pending_messages | session_db_id, status, message_type | CLAIM-CONFIRM queue | +| pending_messages | session_db_id, message_type | Per-session pending queue | | observation_feedback | observation_id, signal_type | Usage tracking | ### ChromaDB (chroma.sqlite3) diff --git a/docs/context/agent-sdk-v2-examples.ts b/docs/context/agent-sdk-v2-examples.ts index 961581d7..b78b06c4 100644 --- a/docs/context/agent-sdk-v2-examples.ts +++ b/docs/context/agent-sdk-v2-examples.ts @@ -1,9 +1,3 @@ -/** - * Claude Agent SDK V2 Examples - * - * The V2 API provides a session-based interface with separate send()/receive(), - * ideal for multi-turn conversations. Run with: npx tsx v2-examples.ts - */ import { unstable_v2_createSession, @@ -32,7 +26,6 @@ async function main() { } } -// Basic session with send/receive pattern async function basicSession() { console.log('=== Basic Session ===\n'); @@ -47,13 +40,11 @@ async function basicSession() { } } -// Multi-turn conversation - V2's key advantage async function multiTurn() { console.log('=== Multi-Turn Conversation ===\n'); await using session = unstable_v2_createSession({ model: 'sonnet' }); - // Turn 1 await session.send('What is 5 + 3? Just the number.'); for await (const msg of session.receive()) { if (msg.type === 'assistant') { @@ -62,7 +53,6 @@ async function multiTurn() { } } - // Turn 2 - Claude remembers context await session.send('Multiply that by 2. Just the number.'); for await (const msg of session.receive()) { if (msg.type === 'assistant') { @@ -72,7 +62,6 @@ async function multiTurn() { } } -// One-shot convenience function async function oneShot() { console.log('=== One-Shot Prompt ===\n'); @@ -84,13 +73,11 @@ async function oneShot() { } } -// Session resume - persist context across sessions async function sessionResume() { console.log('=== Session Resume ===\n'); let sessionId: string | undefined; - // First session - establish a memory { await using session = unstable_v2_createSession({ model: 'sonnet' }); console.log('[Session 1] Telling Claude my favorite color...'); @@ -110,7 +97,6 @@ async function sessionResume() { console.log('--- Session closed. Time passes... ---\n'); - // Resume and verify Claude remembers { await using session = unstable_v2_resumeSession(sessionId!, { model: 'sonnet' }); console.log('[Session 2] Resuming and asking Claude...'); diff --git a/docs/context/hooks-reference-2026-01-07.md b/docs/context/hooks-reference-2026-01-07.md index ac75bf1b..de26fc72 100644 --- a/docs/context/hooks-reference-2026-01-07.md +++ b/docs/context/hooks-reference-2026-01-07.md @@ -332,7 +332,6 @@ Block edits to sensitive files: * For troubleshooting steps and debugging techniques, see [Debugging](/en/hooks#debugging) in the hooks reference documentation. - --- > To find navigation and other pages in this documentation, fetch the llms.txt file at: https://code.claude.com/docs/llms.txt \ No newline at end of file diff --git a/docs/i18n/README.ar.md b/docs/i18n/README.ar.md index fc3bf733..a2f86574 100644 --- a/docs/i18n/README.ar.md +++ b/docs/i18n/README.ar.md @@ -82,7 +82,6 @@

-

بداية سريعة • كيف يعمل • diff --git a/docs/public/architecture/hooks.mdx b/docs/public/architecture/hooks.mdx index c913fe4b..1c1bf7d9 100644 --- a/docs/public/architecture/hooks.mdx +++ b/docs/public/architecture/hooks.mdx @@ -190,13 +190,16 @@ Hooks are configured in `plugin/hooks/hooks.json`: ```json { "hooks": { + "Setup": [{ + "hooks": [{ + "type": "command", + "command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/version-check.js", + "timeout": 60 + }] + }], "SessionStart": [{ "matcher": "startup|clear|compact", "hooks": [{ - "type": "command", - "command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/smart-install.js", - "timeout": 300 - }, { "type": "command", "command": "bun ${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs start", "timeout": 60 @@ -246,9 +249,10 @@ Hooks are configured in `plugin/hooks/hooks.json`: **Timing**: When user opens Claude Code or resumes session **Hooks Triggered** (in order): -1. `smart-install.js` - Ensures dependencies are installed -2. `worker-service.cjs start` - Starts the worker service -3. `context-hook.js` - Fetches and silently injects prior session context +1. `worker-service.cjs start` - Starts the worker service +2. `context-hook.js` - Fetches and silently injects prior session context + +(Runtime setup is handled out-of-band by `npx claude-mem install` / `npx claude-mem repair`. The Setup phase runs a sub-100ms `version-check.js` that prompts the user to repair if the `.install-version` marker is stale.) As of Claude Code 2.1.0 (ultrathink update), SessionStart hooks no longer display user-visible messages. Context is silently injected via `hookSpecificOutput.additionalContext`. diff --git a/docs/public/architecture/overview.mdx b/docs/public/architecture/overview.mdx index 7b2a60b9..4ef33364 100644 --- a/docs/public/architecture/overview.mdx +++ b/docs/public/architecture/overview.mdx @@ -7,30 +7,29 @@ description: "System components and data flow in Claude-Mem" ## System Components -Claude-Mem operates as a Claude Code plugin with five core components: +Claude-Mem operates as a Claude Code plugin with the following core components: -1. **Plugin Hooks** - Capture lifecycle events (6 hook files) -2. **Smart Install** - Cached dependency checker (pre-hook script, runs before context-hook) -3. **Worker Service** - Process observations via Claude Agent SDK + HTTP API (10 search endpoints) -4. **Database Layer** - Store sessions and observations (SQLite + FTS5 + ChromaDB) -5. **mem-search Skill** - Skill-based search with progressive disclosure (v5.4.0+) -6. **Viewer UI** - Web-based real-time memory stream visualization +1. **Plugin Hooks** - Lifecycle events (Setup version-check + 5 lifecycle hooks: SessionStart, UserPromptSubmit, PreToolUse for `Read`, PostToolUse, Stop) +2. **Worker Service** - Express HTTP API on a per-user port; processes observations via the Claude Agent SDK (or Gemini / OpenRouter) +3. **Database Layer** - SQLite + FTS5 (and optional Chroma for semantic search) +4. **Search Tools** - HTTP API + the `mem-search` skill / MCP server for progressive disclosure search +5. **Viewer UI** - React-based real-time memory stream served by the worker ## Technology Stack | Layer | Technology | |------------------------|-------------------------------------------| | **Language** | TypeScript (ES2022, ESNext modules) | -| **Runtime** | Node.js 18+ | +| **Runtime** | Node.js 20+ and Bun ≥ 1.0 | | **Database** | SQLite 3 with bun:sqlite driver | -| **Vector Store** | ChromaDB (optional, for semantic search) | -| **HTTP Server** | Express.js 4.18 | +| **Vector Store** | Chroma (optional, for semantic search) | +| **HTTP Server** | Express.js 5 | | **Real-time** | Server-Sent Events (SSE) | | **UI Framework** | React + TypeScript | -| **AI SDK** | @anthropic-ai/claude-agent-sdk | +| **AI SDK** | @anthropic-ai/claude-agent-sdk (or Gemini / OpenRouter) | | **Build Tool** | esbuild (bundles TypeScript) | | **Process Manager** | Bun | -| **Testing** | Node.js built-in test runner | +| **Testing** | `bun test` | ## Data Flow @@ -63,13 +62,13 @@ Uses 3-layer progressive disclosure: search → timeline → get_observations ``` ┌─────────────────────────────────────────────────────────────────┐ -│ 0. Smart Install Pre-Hook Fires │ -│ Checks dependencies (cached), only runs on version changes │ -│ Not a lifecycle hook - runs before context-hook starts │ +│ 0. Setup Hook Fires (version-check.js) │ +│ Sub-100ms read of .install-version; on mismatch prints │ +│ "run: npx claude-mem repair" to stderr. Always exits 0. │ └─────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ -│ 1. Session Starts → Context Hook Fires │ +│ 1. Session Starts → Worker-start, then Context Hook │ │ Starts Bun worker if needed, injects context from previous │ │ sessions (configurable observation count) │ └─────────────────────────────────────────────────────────────────┘ @@ -106,99 +105,60 @@ Uses 3-layer progressive disclosure: search → timeline → get_observations ``` claude-mem/ ├── src/ -│ ├── hooks/ # Hook implementations (6 hooks) -│ │ ├── context-hook.ts # SessionStart -│ │ ├── user-message-hook.ts # UserMessage (for debugging) -│ │ ├── new-hook.ts # UserPromptSubmit -│ │ ├── save-hook.ts # PostToolUse -│ │ ├── summary-hook.ts # Stop -│ │ ├── cleanup-hook.ts # SessionEnd -│ │ └── hook-response.ts # Hook response utilities -│ │ +│ ├── hooks/ # TypeScript hook implementations (built via esbuild) │ ├── sdk/ # Claude Agent SDK integration -│ │ ├── prompts.ts # XML prompt builders -│ │ ├── parser.ts # XML response parser -│ │ └── worker.ts # Main SDK agent loop -│ │ │ ├── services/ -│ │ ├── worker-service.ts # Express HTTP + SSE service -│ │ └── sqlite/ # Database layer -│ │ ├── SessionStore.ts # CRUD operations -│ │ ├── SessionSearch.ts # FTS5 search service -│ │ ├── migrations.ts -│ │ └── types.ts -│ │ -│ ├── ui/ # Viewer UI -│ │ └── viewer/ # React + TypeScript web interface -│ │ ├── components/ # UI components -│ │ ├── hooks/ # React hooks -│ │ ├── utils/ # Utilities -│ │ └── assets/ # Fonts, logos -│ │ -│ ├── shared/ # Shared utilities -│ │ ├── config.ts -│ │ ├── paths.ts -│ │ └── storage.ts -│ │ -│ └── utils/ -│ ├── logger.ts -│ ├── platform.ts -│ └── port-allocator.ts +│ │ ├── worker-service.ts # Express HTTP + SSE service (worker entry point) +│ │ ├── sync/ChromaSync.ts # Optional Chroma vector index +│ │ └── sqlite/ # SQLite + FTS5 storage layer +│ ├── ui/viewer/ # React + TypeScript web viewer +│ ├── shared/ # Shared utilities (paths, settings defaults) +│ └── utils/ # Logging, platform, tag-stripping helpers │ -├── scripts/ # Build and utility scripts -│ └── smart-install.js # Cached dependency checker (pre-hook) +├── scripts/ # Build + utility scripts │ -├── plugin/ # Plugin distribution -│ ├── .claude-plugin/ -│ │ └── plugin.json -│ ├── hooks/ -│ │ └── hooks.json +├── plugin/ # Plugin distribution (synced to marketplace) +│ ├── .claude-plugin/plugin.json +│ ├── hooks/hooks.json # Hook registration (Setup + 5 lifecycle hooks) │ ├── scripts/ # Built executables -│ │ ├── context-hook.js -│ │ ├── user-message-hook.js -│ │ ├── new-hook.js -│ │ ├── save-hook.js -│ │ ├── summary-hook.js -│ │ ├── cleanup-hook.js -│ │ └── worker-service.cjs # Background worker + HTTP API -│ │ -│ ├── skills/ # Agent skills (v5.4.0+) -│ │ ├── mem-search/ # Search skill with progressive disclosure (v5.5.0) -│ │ │ ├── SKILL.md # Skill frontmatter (~250 tokens) -│ │ │ ├── operations/ # 12 detailed operation docs -│ │ │ └── principles/ # 2 principle guides -│ │ ├── troubleshoot/ # Troubleshooting skill -│ │ │ ├── SKILL.md -│ │ │ └── operations/ # 6 operation docs -│ │ └── version-bump/ # Version management skill (deprecated) -│ │ -│ └── ui/ # Built viewer UI -│ └── viewer.html # Self-contained bundle +│ │ ├── version-check.js # Setup-phase marker check (sub-100ms) +│ │ ├── bun-runner.js # Resolves Bun and runs worker-service.cjs +│ │ ├── worker-service.cjs # Worker daemon + lifecycle hook dispatcher +│ │ ├── worker-cli.js # CLI shim +│ │ ├── worker-wrapper.cjs # Process wrapper +│ │ ├── mcp-server.cjs # MCP search server +│ │ ├── statusline-counts.js +│ │ └── context-generator.cjs +│ ├── skills/ # Agent skills (mem-search, make-plan, do, etc.) +│ └── ui/viewer.html # Self-contained React bundle │ -├── tests/ # Test suite -├── docs/ # Documentation -└── ecosystem.config.cjs # Process configuration (deprecated) +├── tests/ # Test suite (`bun test`) +├── docs/ # Mintlify documentation +└── openclaw/ # OpenClaw integration plugin ``` ## Component Details -### 1. Plugin Hooks (6 Hooks) -- **context-hook.js** - SessionStart: Starts Bun worker, injects context -- **user-message-hook.js** - UserMessage: Debugging hook -- **new-hook.js** - UserPromptSubmit: Creates session, saves prompt -- **save-hook.js** - PostToolUse: Captures tool executions -- **summary-hook.js** - Stop: Generates session summary -- **cleanup-hook.js** - SessionEnd: Marks session complete +### 1. Plugin Hooks -**Note**: smart-install.js is a pre-hook dependency checker (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json and only runs when dependencies need updating. +The plugin registers a Setup-phase `version-check.js` plus five lifecycle hooks. Each lifecycle event invokes `bun-runner.js` to spawn `worker-service.cjs` with a `hook claude-code ` argument; the worker process is the single dispatcher for all hook logic. Events: + +- **Setup** → `version-check.js` (sub-100ms marker check; never installs anything) +- **SessionStart** → start worker, then `hook claude-code context` (context injection) +- **UserPromptSubmit** → `hook claude-code session-init` +- **PreToolUse** (matcher `Read`) → `hook claude-code file-context` +- **PostToolUse** (matcher `*`) → `hook claude-code observation` +- **Stop** → `hook claude-code summarize` (summary generation) + +The actual runtime install (Bun, uv, `bun install`) is performed by `npx claude-mem install` / `npx claude-mem repair` with a visible installer spinner; the Setup hook itself only reads the `.install-version` marker. See [Plugin Hooks](/architecture/hooks) for detailed hook documentation. ### 2. Worker Service -Express.js HTTP server on port 37777 (configurable) with: -- 10 search HTTP API endpoints (v5.4.0+) -- 8 viewer UI HTTP/SSE endpoints -- Async observation processing via Claude Agent SDK +Express.js HTTP server on a per-user port (default `37700 + (uid % 100)`, override via `CLAUDE_MEM_WORKER_PORT`) with: +- Search HTTP API endpoints +- Viewer UI HTTP/SSE endpoints +- Async observation processing via the Claude Agent SDK (or Gemini / OpenRouter) - Real-time updates via Server-Sent Events - Auto-managed by Bun @@ -230,7 +190,7 @@ Skill-based search with progressive disclosure providing 10 search operations: See [Search Architecture](/architecture/search-architecture) for technical details and examples. ### 5. Viewer UI -React + TypeScript web interface at http://localhost:37777 featuring: +React + TypeScript web interface served by the worker on its configured port (default `http://127.0.0.1:`) featuring: - Real-time memory stream via Server-Sent Events - Infinite scroll pagination with automatic deduplication - Project filtering and settings persistence diff --git a/docs/public/architecture/worker-service.mdx b/docs/public/architecture/worker-service.mdx index 49c7ca17..2cbc0453 100644 --- a/docs/public/architecture/worker-service.mdx +++ b/docs/public/architecture/worker-service.mdx @@ -12,10 +12,10 @@ The worker service is a long-running HTTP API built with Express.js and managed - **Technology**: Express.js HTTP server - **Runtime**: Bun (auto-installed if missing) - **Process Manager**: Native Bun process management via ProcessManager -- **Port**: Fixed port 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`) +- **Port**: Per-user default `37700 + (uid % 100)` (override with `CLAUDE_MEM_WORKER_PORT`). The active port is stored in `~/.claude-mem/settings.json` and reported by `GET /api/health`. - **Location**: `src/services/worker-service.ts` - **Built Output**: `plugin/scripts/worker-service.cjs` -- **Model**: Configurable via `CLAUDE_MEM_MODEL` environment variable (default: sonnet) +- **Model**: Configurable via `CLAUDE_MEM_MODEL` (default: `claude-haiku-4-5-20251001`) ## REST API Endpoints @@ -51,10 +51,12 @@ GET /health { "status": "ok", "uptime": 12345, - "port": 37777 + "port": 37742 } ``` +The `port` value is the actual worker port for the current user — per-user default `37700 + (uid % 100)`, or whatever `CLAUDE_MEM_WORKER_PORT` is set to. The example above is illustrative; your value will differ. + #### 3. Server-Sent Events Stream ``` GET /stream @@ -612,7 +614,7 @@ The worker service auto-starts when the SessionStart hook fires. Manual start is ### Bun Requirement -Bun is required to run the worker service. If Bun is not installed, the smart-install script will automatically install it on first run: +Bun is required to run the worker service. If Bun is not installed, `npx claude-mem install` (and `npx claude-mem repair`) installs it globally during setup, with a visible clack spinner: - **Windows**: `powershell -c "irm bun.sh/install.ps1 | iex"` - **macOS/Linux**: `curl -fsSL https://bun.sh/install | bash` @@ -640,26 +642,26 @@ The worker service routes observations to the Claude Agent SDK for AI-powered pr ### Model Configuration -Set the AI model used for processing via environment variable: +Set the Claude model used for compression via environment variable or `~/.claude-mem/settings.json`: ```bash -export CLAUDE_MEM_MODEL=sonnet +export CLAUDE_MEM_MODEL=claude-haiku-4-5-20251001 ``` -Available shorthand models (forward to latest version): -- `haiku` - Fast, cost-efficient -- `sonnet` - Balanced (default) -- `opus` - Most capable +Allowed values: +- `claude-haiku-4-5-20251001` - default, fast and cheap (best for compression) +- `claude-sonnet-4-6` - balanced quality and cost +- `claude-opus-4-7` - highest quality, most expensive ## Port Allocation -The worker uses a fixed port (37777 by default) for consistent communication: +The worker uses a per-user default port so different OS users on the same machine never collide: -- **Default**: Port 37777 -- **Override**: Set `CLAUDE_MEM_WORKER_PORT` environment variable -- **Port File**: `${CLAUDE_PLUGIN_ROOT}/data/worker.port` tracks current port +- **Default**: `37700 + (uid % 100)` (set in `src/shared/SettingsDefaultsManager.ts`) +- **Override**: Set `CLAUDE_MEM_WORKER_PORT` (env or `~/.claude-mem/settings.json`) +- **Discovery**: `GET /api/health` returns the active port; `~/.claude-mem/settings.json` stores the configured value -If port 37777 is in use, the worker will fail to start. Set a custom port via environment variable. +If the chosen port is occupied, the worker fails to start — pin a different port via `CLAUDE_MEM_WORKER_PORT` and restart. ## Data Storage diff --git a/docs/public/beta-features.mdx b/docs/public/beta-features.mdx index db8ee420..b6cbcb5c 100644 --- a/docs/public/beta-features.mdx +++ b/docs/public/beta-features.mdx @@ -13,11 +13,11 @@ Claude-Mem offers a beta channel for users who want to try experimental features ## Version Channel Switching -You can switch between stable and beta versions directly from the web viewer UI at http://localhost:37777. +You can switch between stable and beta versions directly from the web viewer UI (the worker prints its URL on startup; default `http://127.0.0.1:`). ### How to Access -1. Open the Claude-Mem viewer at http://localhost:37777 +1. Open the Claude-Mem viewer (the worker prints its URL on startup) 2. Click the **Settings** gear icon in the top-right 3. Find the **Version Channel** section 4. Click **Try Beta (Endless Mode)** to switch to beta, or **Switch to Stable** to return diff --git a/docs/public/configuration.mdx b/docs/public/configuration.mdx index 2e31a59b..3e31b65f 100644 --- a/docs/public/configuration.mdx +++ b/docs/public/configuration.mdx @@ -13,12 +13,13 @@ Settings are managed in `~/.claude-mem/settings.json`. The file is auto-created | Setting | Default | Description | |-------------------------------|---------------------------------|---------------------------------------| -| `CLAUDE_MEM_MODEL` | `sonnet` | AI model for processing observations (when using Claude) | +| `CLAUDE_MEM_MODEL` | `claude-haiku-4-5-20251001` | Claude model used to compress observations (when using the Claude provider) | | `CLAUDE_MEM_PROVIDER` | `claude` | AI provider: `claude`, `gemini`, or `openrouter` | | `CLAUDE_MEM_MODE` | `code` | Active mode profile (e.g., `code--es`, `email-investigation`) | | `CLAUDE_MEM_CONTEXT_OBSERVATIONS` | `50` | Number of observations to inject | -| `CLAUDE_MEM_WORKER_PORT` | `37777` | Worker service port | +| `CLAUDE_MEM_WORKER_PORT` | `37700 + (uid % 100)` | Worker service port (per-user default; override for fixed port) | | `CLAUDE_MEM_WORKER_HOST` | `127.0.0.1` | Worker service host address | +| `CLAUDE_MEM_DATA_DIR` | `~/.claude-mem` | Data root — every other path (database, chroma, logs, settings.json, worker.pid) derives from this | | `CLAUDE_MEM_SKIP_TOOLS` | `ListMcpResourcesTool,SlashCommand,Skill,TodoWrite,AskUserQuestion` | Comma-separated tools to exclude from observations | ### Gemini Provider Settings @@ -54,23 +55,19 @@ See [OpenRouter Provider](usage/openrouter-provider) for detailed configuration, ## Model Configuration -Configure which AI model processes your observations. +Configure which Claude model compresses your observations (only applies when `CLAUDE_MEM_PROVIDER=claude`). ### Available Models -Shorthand model names automatically forward to the latest version: +| Value | Notes | +|-------|-------| +| `claude-haiku-4-5-20251001` | Default — fast and cheap, ideal for compression | +| `claude-sonnet-4-6` | Balanced quality and cost | +| `claude-opus-4-7` | Highest quality, most expensive | -- `haiku` - Fast, cost-efficient -- `sonnet` - Balanced (default) -- `opus` - Most capable +### Picking via the Installer -### Using the Interactive Script - -```bash -./claude-mem-settings.sh -``` - -This script manages settings in `~/.claude-mem/settings.json`. +`npx claude-mem install` prompts for the Claude model (when the Claude provider is selected) and persists the choice to `~/.claude-mem/settings.json`. ### Manual Configuration @@ -78,7 +75,7 @@ Edit `~/.claude-mem/settings.json`: ```json { - "CLAUDE_MEM_MODEL": "sonnet" + "CLAUDE_MEM_MODEL": "claude-haiku-4-5-20251001" } ``` @@ -119,8 +116,8 @@ The data directory location depends on the environment: ``` ~/.claude-mem/ ├── claude-mem.db # SQLite database -├── .install-version # Cached version for smart installer -├── worker.port # Current worker port file +├── .install-version # Version marker written by `npx claude-mem install`/`repair` +├── settings.json # Worker port + provider/model settings └── logs/ ├── worker-out.log # Worker stdout logs └── worker-error.log # Worker stderr logs @@ -136,7 +133,7 @@ ${CLAUDE_PLUGIN_ROOT}/ ├── hooks/ │ └── hooks.json # Hook configuration ├── scripts/ # Built executables -│ ├── smart-install.js # Smart installer script +│ ├── version-check.js # Sub-100ms Setup-hook version marker check │ ├── context-hook.js # Context injection hook │ ├── new-hook.js # Session creation hook │ ├── save-hook.js # Observation capture hook @@ -151,44 +148,16 @@ ${CLAUDE_PLUGIN_ROOT}/ ### Hooks Configuration -Hooks are configured in `plugin/hooks/hooks.json`: +Hooks are registered in `plugin/hooks/hooks.json`. The current shape uses a single dispatcher (`worker-service.cjs hook claude-code `) launched through `bun-runner.js`, plus a fast Setup-phase `version-check.js`. The events wired up are: -```json -{ - "description": "Claude-mem memory system hooks", - "hooks": { - "SessionStart": [{ - "hooks": [{ - "type": "command", - "command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/smart-install.js && node ${CLAUDE_PLUGIN_ROOT}/scripts/context-hook.js", - "timeout": 120 - }] - }], - "UserPromptSubmit": [{ - "hooks": [{ - "type": "command", - "command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/new-hook.js", - "timeout": 120 - }] - }], - "PostToolUse": [{ - "matcher": "*", - "hooks": [{ - "type": "command", - "command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/save-hook.js", - "timeout": 120 - }] - }], - "Stop": [{ - "hooks": [{ - "type": "command", - "command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js", - "timeout": 120 - }] - }] - } -} -``` +- `Setup` → `version-check.js` (sub-100ms `.install-version` check) +- `SessionStart` → start the worker, then `hook claude-code context` (context injection) +- `UserPromptSubmit` → `hook claude-code session-init` +- `PreToolUse` (matcher `Read`) → `hook claude-code file-context` +- `PostToolUse` (matcher `*`) → `hook claude-code observation` +- `Stop` → `hook claude-code summarize` + +The exact `hooks.json` entries are written by the installer; do not hand-edit them in the marketplace copy unless you know what you're doing. ### Search Configuration @@ -198,7 +167,7 @@ Claude-Mem provides MCP search tools for querying your project history. Search operations are provided via: - **MCP Server**: 3 tools (search, timeline, get_observations) with progressive disclosure -- **HTTP API**: 10 endpoints on worker service port 37777 +- **HTTP API**: 10 endpoints on the worker service port (per-user, default `37700 + (uid % 100)`; see `~/.claude-mem/settings.json`) - **Auto-Invocation**: Claude recognizes natural language queries about past work ## Version Channel @@ -207,7 +176,7 @@ Claude-Mem supports switching between stable and beta versions via the web viewe ### Accessing Version Channel -1. Open the viewer at http://localhost:37777 +1. Open the viewer at the worker URL (default `http://127.0.0.1:`; the active port is the value of `CLAUDE_MEM_WORKER_PORT` in `~/.claude-mem/settings.json`) 2. Click the Settings gear icon 3. Find the **Version Channel** section @@ -243,7 +212,7 @@ Claude-Mem injects past observations into each new session, giving Claude awaren ### Context Settings Modal -Access the settings modal from the web viewer at http://localhost:37777: +Access the settings modal from the web viewer (the worker prints its URL on startup; default is `http://127.0.0.1:`): 1. Click the **gear icon** in the header 2. Adjust settings in the right panel @@ -315,7 +284,7 @@ Token economics help you understand the value of cached observations vs. re-read | Setting | Default | Description | |---------|---------|-------------| | **Model** | sonnet | AI model for generating observations | -| **Worker Port** | 37777 | Port for background worker service | +| **Worker Port** | `37700 + (uid % 100)` | Port for background worker service (override with `CLAUDE_MEM_WORKER_PORT`) | | **MCP search server** | true | Enable Model Context Protocol search tools | | **Include last summary** | false | Add previous session's summary to context | | **Include last message** | false | Add previous session's final message | @@ -340,7 +309,7 @@ Settings are stored in `~/.claude-mem/settings.json`: } ``` -**Note**: The Context Settings Modal (at http://localhost:37777) is the recommended way to configure these settings, as it provides live preview of changes. +**Note**: The Context Settings Modal (in the web viewer) is the recommended way to configure these settings, as it provides live preview of changes. ## Customization @@ -411,22 +380,16 @@ Changes take effect on the next tool execution (no worker restart needed). ### Hook Timeouts -Modify timeouts in `plugin/hooks/hooks.json`: +Hook timeouts are written into `plugin/hooks/hooks.json` by the installer. The current defaults match the shape of the workload at each lifecycle stage: -```json -{ - "timeout": 120 // Default: 120 seconds -} -``` - -Recommended values: -- SessionStart: 120s (needs time for smart install check and context retrieval) +- Setup (`version-check.js`): 300s ceiling but normally < 100ms — only reads `.install-version` +- SessionStart (worker-start + context): 60s - UserPromptSubmit: 60s -- PostToolUse: 120s (can process many observations) -- Stop: 60s -- SessionEnd: 60s +- PreToolUse (file-context, Read matcher): 60s +- PostToolUse (observation): 120s +- Stop (summary): 120s -**Note**: With smart install caching (v5.0.3+), SessionStart is typically very fast (10ms) unless dependencies need installation. +The Setup hook never installs anything — runtime install (Bun, uv, `bun install`) happens in `npx claude-mem install` / `npx claude-mem repair` outside the session lifecycle. ### Worker Memory Limit @@ -472,16 +435,15 @@ npm run worker:logs ### Invalid Model Name -If you specify an invalid model name, the worker will fall back to `sonnet` and log a warning. +If you specify an invalid Claude model name, the worker logs a warning and uses the default. Valid Claude models for `CLAUDE_MEM_MODEL`: -Valid shorthand models (forward to latest version): -- haiku -- sonnet -- opus +- `claude-haiku-4-5-20251001` (default) +- `claude-sonnet-4-6` +- `claude-opus-4-7` ### Port Already in Use -If port 37777 is already in use: +The default worker port is `37700 + (uid % 100)`, so different OS users on the same machine get different ports automatically. If you still hit a collision (e.g. running multiple profiles as the same UID), set a fixed port: 1. Set custom port: ```bash @@ -495,7 +457,7 @@ If port 37777 is already in use: 3. Verify new port: ```bash - cat ~/.claude-mem/worker.port + curl -s http://127.0.0.1:$CLAUDE_MEM_WORKER_PORT/api/health | jq .port ``` ## Next Steps diff --git a/docs/public/development.mdx b/docs/public/development.mdx index 03546bd1..b0dd3496 100644 --- a/docs/public/development.mdx +++ b/docs/public/development.mdx @@ -39,7 +39,7 @@ The build process uses esbuild to compile TypeScript: **Build Output**: - Hook executables: `*-hook.js` (ESM format) -- Smart installer: `smart-install.js` (ESM format) +- Setup version-check: `version-check.js` (ESM format, sub-100ms) - Worker service: `worker-service.cjs` (CJS format) - MCP server: `mcp-server.cjs` (CJS format) - Viewer UI: `viewer.html` (self-contained HTML bundle) @@ -320,7 +320,7 @@ npm test export function buildObservationPrompt(observation: Observation): string { return ` - + `; } diff --git a/docs/public/hooks-architecture.mdx b/docs/public/hooks-architecture.mdx index 8115f29b..4b072af3 100644 --- a/docs/public/hooks-architecture.mdx +++ b/docs/public/hooks-architecture.mdx @@ -54,7 +54,7 @@ Claude Code's hook system provides exactly what we need: - SessionStart, UserPromptSubmit, PostToolUse, Stop + SessionStart, UserPromptSubmit, PreToolUse (Read), PostToolUse, Stop, SessionEnd @@ -74,34 +74,31 @@ Claude Code's hook system provides exactly what we need: ## The Hook Scripts -Claude-Mem uses lifecycle hook scripts across 5 lifecycle events. SessionStart runs 3 hooks in sequence: smart-install, worker-service start, and context-hook. +Claude-Mem uses lifecycle hook scripts across 5 lifecycle events. Runtime setup is handled out-of-band by `npx claude-mem install` (and `npx claude-mem repair`); the Setup hook only runs a sub-100ms `version-check.js` to flag stale installs. SessionStart runs 2 hooks in sequence: worker-service start, then context-hook. -### Pre-Hook: Smart Install (Before SessionStart) +### Setup Hook: Version Check -**Purpose:** Intelligently manage dependencies and start worker service +**Purpose:** Detect stale installs caused by external plugin upgrades and prompt the user to repair. -**Note:** This is NOT a lifecycle hook - it's a pre-hook script executed via command chaining before context-hook runs. +**Note:** Runtime installation (Bun, uv, `bun install` in the plugin cache) is performed by `npx claude-mem install` and `npx claude-mem repair` — the Setup hook itself never installs anything. -**When:** Claude Code starts (startup, clear, or compact) +**When:** Claude Code Setup phase, before every session. **What it does:** -1. Checks if dependencies need installation (version marker) -2. Only runs `npm install` when necessary: - - First-time installation - - Version changed in package.json -3. Provides Windows-specific error messages -4. Starts Bun worker service +1. Reads the `.install-version` marker written by the npx installer. +2. Compares it against the currently loaded plugin version. +3. On mismatch, writes `run: npx claude-mem repair` to stderr. +4. Always exits 0 (non-blocking, sub-100ms). **Configuration:** ```json { "hooks": { - "SessionStart": [{ - "matcher": "startup|clear|compact", + "Setup": [{ "hooks": [{ "type": "command", - "command": "node \"${CLAUDE_PLUGIN_ROOT}/../scripts/smart-install.js\" && node ${CLAUDE_PLUGIN_ROOT}/scripts/context-hook.js", - "timeout": 300 + "command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/version-check.js", + "timeout": 60 }] }] } @@ -109,14 +106,11 @@ Claude-Mem uses lifecycle hook scripts across 5 lifecycle events. SessionStart r ``` **Key Features:** -- ✅ Version caching (`.install-version` file) -- ✅ Fast when already installed (~10ms vs 2-5 seconds) -- ✅ Cross-platform compatible -- ✅ Helpful Windows error messages for build tools +- ✅ Sub-100ms version-marker check (no I/O beyond reading the marker) +- ✅ Always exit 0 — never blocks a session +- ✅ Clear repair instructions on stderr when the plugin was upgraded externally (e.g. `claude plugin update`) -**v5.0.3 Enhancement:** Smart caching eliminates redundant installs - -**Source:** `scripts/smart-install.js` +**Source:** `scripts/version-check.js`. The matching installer logic lives in `npx claude-mem install` / `npx claude-mem repair`, which install Bun + uv globally, run `bun install` in the plugin cache, and write the `.install-version` marker — all behind a visible clack spinner. --- @@ -124,7 +118,7 @@ Claude-Mem uses lifecycle hook scripts across 5 lifecycle events. SessionStart r **Purpose:** Inject relevant context from previous sessions -**When:** Claude Code starts (runs after smart-install pre-hook) +**When:** Claude Code starts (runs after the worker-start SessionStart entry) **What it does:** 1. Extracts project name from current working directory @@ -429,7 +423,7 @@ sequenceDiagram | Event | Timing | Blocking | Timeout | Output Handling | |-------|--------|----------|---------|-----------------| -| **SessionStart (smart-install)** | Before session | No | 300s | stderr (log only) | +| **Setup (version-check)** | Before session | No | 60s | stderr hint on stale install (always exit 0) | | **SessionStart (worker-start)** | Before session | No | 60s | stderr (log only) | | **SessionStart (context)** | Before session | No | 60s | JSON → additionalContext (silent) | | **UserPromptSubmit** | Before processing | No | 60s | stdout → context | @@ -501,7 +495,7 @@ npm run worker:stop ### Worker HTTP API -**Technology:** Express.js REST API on port 37777 +**Technology:** Express.js REST API on the worker's per-user port (default `37700 + (uid % 100)`, override via `CLAUDE_MEM_WORKER_PORT`) **Endpoints:** @@ -692,23 +686,18 @@ claude --debug | Hook | Average | p95 | p99 | |------|---------|-----|-----| -| SessionStart (smart-install, cached) | 10ms | 20ms | 40ms | -| SessionStart (smart-install, first run) | 2500ms | 5000ms | 8000ms | +| Setup (version-check, marker matches) | 8ms | 20ms | 40ms | +| Setup (version-check, marker mismatch — stderr hint, still non-blocking) | 10ms | 25ms | 50ms | | SessionStart (context) | 45ms | 120ms | 250ms | | SessionStart (user-message) | 5ms | 10ms | 15ms | | UserPromptSubmit | 12ms | 25ms | 50ms | | PostToolUse | 8ms | 15ms | 30ms | | SessionEnd | 5ms | 10ms | 20ms | -**Why smart-install is sometimes slow:** -- First-time: Full npm install (2-5 seconds) -- Cached: Version check only (~10ms) -- Version change: Full npm install + worker restart - -**Optimization (v5.0.3):** -- Version caching with `.install-version` marker -- Only install on version change or missing deps -- Windows-specific error messages with build tool help +**Why the Setup hook stays fast:** +- The Setup hook only reads the `.install-version` marker — no `npm install`, no spawned subprocesses. +- All heavy lifting (Bun + uv install, `bun install` inside the plugin cache) happens in `npx claude-mem install` / `npx claude-mem repair`, which run with a visible clack spinner outside the session lifecycle. +- On marker mismatch the hook prints a one-line `run: npx claude-mem repair` hint to stderr and exits 0; the user opts into the slow path explicitly. ### Database Performance diff --git a/docs/public/installation.mdx b/docs/public/installation.mdx index 0db50280..2fdd2527 100644 --- a/docs/public/installation.mdx +++ b/docs/public/installation.mdx @@ -16,10 +16,12 @@ npx claude-mem install ``` The interactive installer will: -- Detect your installed IDEs (Claude Code, Cursor, Gemini CLI, Windsurf, etc.) -- Copy plugin files to the correct locations -- Register the plugin with Claude Code -- Install all dependencies (including Bun and uv) +- Run a runtime check (auto-installs Bun and uv if missing) +- Detect your installed IDEs (Claude Code, Cursor, Gemini CLI, Windsurf, OpenCode, Codex CLI) and let you multi-select which ones to wire up +- Offer to install Claude Code if it isn't found +- Prompt for an LLM provider (Claude Code auth, Gemini API key, or OpenRouter API key) +- Prompt for the Claude model used to compress observations (Haiku / Sonnet / Opus) when the Claude provider is selected +- Copy plugin files into the marketplace directory and register the plugin - Auto-start the worker service ### Option 2: Plugin Marketplace @@ -39,10 +41,11 @@ Both methods will automatically configure hooks and start the worker service. St ## System Requirements -- **Node.js**: 18.0.0 or higher -- **Claude Code**: Latest version with plugin support -- **Bun**: JavaScript runtime and process manager (auto-installed if missing) -- **SQLite 3**: For persistent storage (bundled) +- **Node.js**: 20.0.0 or higher +- **Bun** ≥ 1.0 (auto-installed by `npx claude-mem install` if missing) +- **uv** (auto-installed if missing — provides Python for Chroma's embedding service) +- **Claude Code** or another supported IDE (Cursor, Gemini CLI, Windsurf, OpenCode, Codex CLI, OpenClaw) +- **SQLite 3**: bundled via `bun:sqlite` ## Advanced Installation @@ -73,7 +76,7 @@ npm run worker:status #### 1. Automatic Dependency Installation -Dependencies are installed automatically during plugin installation. The SessionStart hook also ensures dependencies are up-to-date on each session start (this is fast and idempotent). Works cross-platform on Windows, macOS, and Linux. +Dependencies are installed automatically by `npx claude-mem install` and `npx claude-mem repair`. Heavy lifting (Bun + uv install, `bun install` inside the plugin cache) happens behind a visible installer spinner. The Setup hook only performs a sub-100ms `version-check.js` read of the `.install-version` marker — on mismatch it prints `run: npx claude-mem repair` to stderr and exits 0, so it never blocks a session. Works cross-platform on Windows, macOS, and Linux. #### 2. Verify Plugin Installation @@ -110,13 +113,7 @@ npm run test:context ## Upgrading -Upgrades are automatic when updating via the plugin marketplace. Key changes in recent versions: - -**v7.1.0**: PM2 replaced with native Bun process management. Migration is automatic on first hook trigger. - -**v7.0.0+**: 11 configuration settings, dual-tag privacy system. - -**v5.4.0+**: Skill-based search replaces MCP tools, saving ~2,250 tokens per session. +Upgrades are automatic when updating via the plugin marketplace. After an external upgrade (for example `claude plugin update`), the Setup hook detects a version-marker mismatch and asks you to run `npx claude-mem repair`, which installs any missing runtime dependencies and refreshes the marker. See [CHANGELOG](https://github.com/thedotmack/claude-mem/blob/main/CHANGELOG.md) for complete version history. diff --git a/docs/public/introduction.mdx b/docs/public/introduction.mdx index 1e1f641f..c42c3d02 100644 --- a/docs/public/introduction.mdx +++ b/docs/public/introduction.mdx @@ -34,7 +34,7 @@ Restart Claude Code. Context from previous sessions will automatically appear in - 🎭 **Mode System** - Switch between workflows (Code, Email Investigation, Chill) - 🔍 **MCP Search Tools** - Query your project history with natural language - 🧠 **Knowledge Agents** - Build queryable "brains" from your observation history -- 🌐 **Web Viewer UI** - Real-time memory stream visualization at http://localhost:37777 +- 🌐 **Web Viewer UI** - Real-time memory stream visualization served by the local worker - 🔒 **Privacy Control** - Use `` tags to exclude sensitive content from storage - ⚙️ **Context Configuration** - Fine-grained control over what context gets injected - 🤖 **Automatic Operation** - No manual intervention required @@ -66,40 +66,22 @@ Restart Claude Code. Context from previous sessions will automatically appear in ``` **Core Components:** -1. **4 Lifecycle Hooks** - SessionStart, UserPromptSubmit, PostToolUse, Stop -2. **Smart Install** - Cached dependency checker (pre-hook script) -3. **Worker Service** - HTTP API on port 37777 managed by Bun -4. **SQLite Database** - Stores sessions, observations, summaries with FTS5 search -5. **MCP Search Tools** - Query historical context with natural language +1. **5 Lifecycle Hooks** - SessionStart, UserPromptSubmit, PostToolUse, Summary (Stop), SessionEnd +2. **Worker Service** - Express HTTP API managed by Bun on a per-user port (default `37700 + (uid % 100)`) +3. **SQLite Database** - Stores sessions, observations, summaries with FTS5 search +4. **Chroma Vector Index** - Optional embedding-based semantic search +5. **MCP / Skill Search Tools** - Query historical context with natural language 6. **Web Viewer UI** - Real-time visualization with SSE and infinite scroll See [Architecture Overview](architecture/overview) for details. ## System Requirements -- **Node.js**: 18.0.0 or higher -- **Claude Code**: Latest version with plugin support -- **Bun**: JavaScript runtime and process manager (auto-installed if missing) -- **SQLite 3**: For persistent storage (bundled) - -## What's New - -**v9.0.0 - Live Context:** -- **Folder Context Files**: Auto-generated `CLAUDE.md` in project folders with activity timelines -- **Worktree Support**: Unified context from parent repos and git worktrees -- **Configurable Observation Limits**: Control how many observations appear in context -- **Windows Fixes**: Resolved IPC detection and hook execution issues -- **Settings Auto-Creation**: `settings.json` now auto-creates on first run -- **MCP Tools Naming**: Updated from "mem-search skill" to "MCP tools" terminology - -**v7.1.0 - Bun Migration:** -- Replaced PM2 with native Bun process management -- Switched from better-sqlite3 to bun:sqlite for faster database access -- Simplified cross-platform support - -**v7.0.0 - Context Configuration:** -- 11 settings for fine-grained control over context injection -- Dual-tag privacy system (`` tags) +- **Node.js**: 20.0.0 or higher +- **Bun** ≥ 1.0 (auto-installed by `npx claude-mem install` if missing) +- **uv** (auto-installed if missing — provides Python for Chroma) +- **Claude Code** (or another supported IDE: Cursor, Gemini CLI, Windsurf, OpenCode, Codex CLI, OpenClaw) +- **SQLite 3** — bundled via `bun:sqlite` ## Next Steps diff --git a/docs/public/platform-integration.mdx b/docs/public/platform-integration.mdx index a9d06846..8ea39d5f 100644 --- a/docs/public/platform-integration.mdx +++ b/docs/public/platform-integration.mdx @@ -5,8 +5,7 @@ icon: plug --- -**Version:** 7.0.0 (December 2025) -**Target Audience:** Developers building claude-mem integrations (VSCode extensions, IDE plugins, CLI tools) +**Target Audience:** Developers building claude-mem integrations (VSCode extensions, IDE plugins, CLI tools). ## Quick Reference @@ -14,8 +13,11 @@ icon: plug ### Worker Service Basics ```typescript -const WORKER_BASE_URL = 'http://localhost:37777'; -const DEFAULT_PORT = 37777; // Override with CLAUDE_MEM_WORKER_PORT +// Resolve the worker port at runtime. The default is per-user (37700 + uid % 100), +// or whatever the user set via CLAUDE_MEM_WORKER_PORT / settings.json. Read it from +// process.env.CLAUDE_MEM_WORKER_PORT, then ~/.claude-mem/settings.json +// (CLAUDE_MEM_WORKER_PORT key), then fall back to the deterministic default. +const WORKER_BASE_URL = `http://127.0.0.1:${workerPort}`; ``` ### Most Common Operations @@ -46,9 +48,10 @@ GET /api/context/recent?project=my-project&limit=3 ### Environment Variables ```bash -CLAUDE_MEM_MODEL=claude-sonnet-4-6 # Model for observations/summaries +CLAUDE_MEM_MODEL=claude-haiku-4-5-20251001 # Default Claude model for observations/summaries CLAUDE_MEM_CONTEXT_OBSERVATIONS=50 # Observations injected at SessionStart -CLAUDE_MEM_WORKER_PORT=37777 # Worker service port +CLAUDE_MEM_WORKER_PORT= # Optional override; default = 37700 + (uid % 100) +CLAUDE_MEM_DATA_DIR= # Optional override for the data directory CLAUDE_MEM_PYTHON_VERSION=3.13 # Python version for chroma-mcp ``` @@ -68,7 +71,7 @@ npm run worker:status # Check worker status ```plaintext Platform Hook/Extension - → HTTP Request to Worker (localhost:37777) + → HTTP Request to Worker (`${WORKER_BASE_URL}` — per-user, default 37700+uid%100) → Route Handler (SessionRoutes/DataRoutes/SearchRoutes/etc.) → Domain Service (SessionManager/SearchManager/DatabaseManager) → Database (SQLite3 + Chroma vector DB) @@ -311,7 +314,7 @@ GET /api/stats "uptime": 12345, "activeSessions": 2, "sseClients": 1, - "port": 37777 + "port": 37742 }, "database": { "path": "~/.claude-mem/claude-mem.db", @@ -796,7 +799,7 @@ interface TimelineItem { async function onPostToolUse(context: HookContext) { const { session_id, tool_name, tool_input, tool_result, cwd } = context; - const response = await fetch('http://localhost:37777/api/sessions/observations', { + const response = await fetch(`${WORKER_BASE_URL}/api/sessions/observations`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ @@ -818,7 +821,7 @@ interface TimelineItem { async function onSummary(context: HookContext) { const { session_id, last_user_message, last_assistant_message } = context; - await fetch('http://localhost:37777/api/sessions/summarize', { + await fetch(`${WORKER_BASE_URL}/api/sessions/summarize`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ @@ -835,7 +838,7 @@ interface TimelineItem { async function onSessionEnd(context: HookContext) { const { session_id } = context; - await fetch('http://localhost:37777/api/sessions/complete', { + await fetch(`${WORKER_BASE_URL}/api/sessions/complete`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ @@ -867,7 +870,7 @@ const searchTool: SearchTool = { try { const response = await fetch( - `http://localhost:37777/api/search?query=${encodeURIComponent(query)}&format=index&limit=10` + `${WORKER_BASE_URL}/api/search?query=${encodeURIComponent(query)}&format=index&limit=10` ); if (!response.ok) { @@ -905,7 +908,7 @@ const participant = vscode.chat.createChatParticipant('claude-mem', async (reque stream.markdown(`Searching memory for: ${request.prompt}\n\n`); const response = await fetch( - `http://localhost:37777/api/search?query=${encodeURIComponent(request.prompt)}&format=index&limit=5` + `${WORKER_BASE_URL}/api/search?query=${encodeURIComponent(request.prompt)}&format=index&limit=5` ); const results = await response.json(); @@ -931,7 +934,7 @@ async function callWorkerWithFallback( options?: RequestInit ): Promise { try { - const response = await fetch(`http://localhost:37777${endpoint}`, { + const response = await fetch(`${WORKER_BASE_URL}${endpoint}`, { ...options, signal: AbortSignal.timeout(5000) // 5s timeout }); @@ -975,7 +978,7 @@ async function retryWithBackoff( ```typescript async function isWorkerHealthy(): Promise { try { - const response = await fetch('http://localhost:37777/api/health', { + const response = await fetch(`${WORKER_BASE_URL}/api/health`, { signal: AbortSignal.timeout(2000) }); return response.ok; @@ -1017,7 +1020,7 @@ class WorkerTimeoutError extends Error { ```typescript function connectToSSE(onEvent: (event: any) => void) { - const eventSource = new EventSource('http://localhost:37777/stream'); + const eventSource = new EventSource(`${WORKER_BASE_URL}/stream`); eventSource.onmessage = (event) => { try { @@ -1138,8 +1141,8 @@ esbuild.build({ ```bash - curl http://localhost:37777/api/health - curl "http://localhost:37777/api/search?query=test&limit=5" + curl http://127.0.0.1:$WORKER_PORT/api/health + curl "http://127.0.0.1:$WORKER_PORT/api/search?query=test&limit=5" ``` @@ -1239,8 +1242,8 @@ describe('Worker Integration', () => { - [ ] Worker starts successfully (`npm run worker:status`) - - [ ] Health endpoint responds (`curl http://localhost:37777/api/health`) - - [ ] SSE stream connects (`curl http://localhost:37777/stream`) + - [ ] Health endpoint responds (`curl http://127.0.0.1:$WORKER_PORT/api/health`) + - [ ] SSE stream connects (`curl http://127.0.0.1:$WORKER_PORT/stream`) @@ -1284,8 +1287,11 @@ describe('Worker Integration', () => { export class WorkerClient { private baseUrl: string; - constructor(port: number = 37777) { - this.baseUrl = `http://localhost:${port}`; + // Resolve the active worker port via env / settings.json, falling back + // to the deterministic per-user default. See parseWorkerPort() for an + // example helper; never hardcode a single value. + constructor(port: number = resolveWorkerPort()) { + this.baseUrl = `http://127.0.0.1:${port}`; } async isHealthy(): Promise { diff --git a/docs/public/troubleshooting.mdx b/docs/public/troubleshooting.mdx index 826a7793..55eeb52c 100644 --- a/docs/public/troubleshooting.mdx +++ b/docs/public/troubleshooting.mdx @@ -21,24 +21,24 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de --- -## v5.x Specific Issues +## Common Issues ### Viewer UI Not Loading -**Symptoms**: Cannot access http://localhost:37777, page doesn't load, or shows connection error. +**Symptoms**: Cannot reach the viewer URL, page doesn't load, or browser shows a connection error. **Solutions**: -1. Check if worker is running on port 37777: +1. Find the worker port. The default is `37700 + (uid % 100)`. The configured port is the value of `CLAUDE_MEM_WORKER_PORT` in `~/.claude-mem/settings.json`; the running worker also reports it on `/api/health`: ```bash - lsof -i :37777 - # or + PORT=$(jq -r .CLAUDE_MEM_WORKER_PORT ~/.claude-mem/settings.json) + lsof -i :$PORT npm run worker:status ``` 2. Verify worker is healthy: ```bash - curl http://localhost:37777/health + curl http://127.0.0.1:$PORT/health ``` 3. Check worker logs for errors: @@ -51,9 +51,8 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de npm run worker:restart ``` -5. Check for port conflicts: +5. Pin a fixed port if the auto-assigned one collides: ```bash - # If port 37777 is in use by another service export CLAUDE_MEM_WORKER_PORT=38000 npm run worker:restart ``` @@ -170,7 +169,6 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de 4. Restart Claude Code after manual install - ## Worker Service Issues ### Worker Service Not Starting @@ -230,7 +228,7 @@ The skill includes comprehensive diagnostics, automated repair sequences, and de 4. Verify new port: ```bash - cat ~/.claude-mem/worker.port + curl -s http://127.0.0.1:$CLAUDE_MEM_WORKER_PORT/api/health | jq .port ``` ### Worker Keeps Crashing @@ -954,11 +952,11 @@ npm run worker:status # View logs npm run worker:logs -# Check port file -cat ~/.claude-mem/worker.port +# Check configured port (per-user default = 37700 + uid % 100) +jq -r .CLAUDE_MEM_WORKER_PORT ~/.claude-mem/settings.json -# Test worker health -curl http://localhost:37777/health +# Test worker health (substitute PORT for the value above) +curl "http://127.0.0.1:$PORT/health" ``` ### Database Inspection diff --git a/docs/public/usage/folder-context.mdx b/docs/public/usage/folder-context.mdx index 24d6b8f5..6a1df29a 100644 --- a/docs/public/usage/folder-context.mdx +++ b/docs/public/usage/folder-context.mdx @@ -34,8 +34,6 @@ Each folder's `CLAUDE.md` contains a "Recent Activity" section showing: # Recent Activity - - ### Jan 4, 2026 | ID | Time | T | Title | Read | diff --git a/docs/public/usage/getting-started.mdx b/docs/public/usage/getting-started.mdx index 18b30f32..aa921b24 100644 --- a/docs/public/usage/getting-started.mdx +++ b/docs/public/usage/getting-started.mdx @@ -76,7 +76,7 @@ This means Claude "remembers" what happened in previous sessions! ### Worker Management -v4.0+ auto-starts the worker on first session. Manual commands below are optional. +The worker auto-starts on the first SessionStart hook, so you usually don't need these commands. They're handy for diagnostics. ```bash # Start worker service (optional - auto-starts automatically) diff --git a/evals/swebench/Dockerfile.agent b/evals/swebench/Dockerfile.agent index edfc16a0..35d1eb28 100644 --- a/evals/swebench/Dockerfile.agent +++ b/evals/swebench/Dockerfile.agent @@ -1,19 +1,8 @@ -# claude-mem SWE-bench agent image -# Plan: .claude/plans/swebench-claude-mem-docker.md (Phase 1) -# -# Produces `claude-mem/swebench-agent:latest`: Claude Code CLI 2.1.114 + -# locally-built claude-mem plugin, ready to run headlessly per SWE-bench -# instance. Auth (ANTHROPIC_API_KEY) is passed at runtime, never baked in. FROM node:20-bookworm-slim ENV DEBIAN_FRONTEND=noninteractive -# System dependencies: -# git, curl, ca-certificates, unzip — base tooling (Bun installer needs unzip) -# jq — JSONL assembly in run-instance.sh -# uuid-runtime — uuidgen for per-instance session IDs (Phase 2) -# sqlite3 — verifies the claude-mem observations DB RUN apt-get update \ && apt-get install -y --no-install-recommends \ git \ @@ -25,46 +14,25 @@ RUN apt-get update \ sqlite3 \ && rm -rf /var/lib/apt/lists/* -# Bun (claude-mem worker service runs under Bun). Installed to a system -# location so the non-root runtime user can execute it. ENV BUN_INSTALL="/usr/local/bun" RUN curl -fsSL https://bun.sh/install | bash \ && chmod -R a+rX /usr/local/bun ENV PATH="/usr/local/bun/bin:${PATH}" -# uv (provides Python for Chroma per CLAUDE.md). Installed to a system -# location, same reason. ENV UV_INSTALL_DIR="/usr/local/bin" -# Group the chmod so the trailing `|| true` only absorbs chmod failures; without -# this grouping, bash precedence (`&&` binds tighter than `||`) would silently -# mask a failed `curl|sh` install step. RUN set -eux \ && curl -LsSf https://astral.sh/uv/install.sh | sh \ && { chmod a+rX /usr/local/bin/uv /usr/local/bin/uvx 2>/dev/null || true; } -# Claude Code CLI — PINNED to the version whose flag surface was verified in -# the plan (Phase 0). Do NOT bump without re-verifying flags. RUN npm install -g @anthropic-ai/claude-code@2.1.114 -# Locally-built claude-mem plugin. The build-agent-image.sh wrapper runs -# `npm run build` before `docker build`, so plugin/ is populated in the build -# context. We do NOT install claude-mem from npm — we want the current -# worktree under test. COPY plugin/ /opt/claude-mem/ -# Runner script — entrypoint for per-instance invocation (Phase 2 deliverable). COPY evals/swebench/run-instance.sh /evals/swebench/run-instance.sh RUN chmod +x /evals/swebench/run-instance.sh -# Pre-create per-instance config dirs. run-instance.sh overrides HOME to a -# scratch dir for isolation, but having these present keeps tools from -# bailing if they probe the default locations before HOME is set. RUN mkdir -p /root/.claude /root/.claude-mem -# Non-root user. Claude Code refuses `--dangerously-skip-permissions` / -# `--permission-mode bypassPermissions` when euid==0 as a safety rail, so we -# need an unprivileged user for headless batch runs. node:20 already ships a -# `node` user at uid 1000 — reuse it. RUN mkdir -p /home/node/.claude /home/node/.claude-mem \ && chown -R node:node /home/node /opt/claude-mem diff --git a/evals/swebench/build-agent-image.sh b/evals/swebench/build-agent-image.sh index 847f6a96..bcd7708a 100755 --- a/evals/swebench/build-agent-image.sh +++ b/evals/swebench/build-agent-image.sh @@ -1,19 +1,13 @@ #!/usr/bin/env bash -# Build the claude-mem SWE-bench agent image. -# Plan: .claude/plans/swebench-claude-mem-docker.md (Phase 1, step 2) set -euo pipefail -# Resolve repo root (two levels up from this script: evals/swebench -> repo). SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" cd "$REPO_ROOT" -# 1. Build the plugin so plugin/ is populated for the COPY step in the Dockerfile. npm run build -# 2. Build the agent image. Context is the repo root so both plugin/ and -# evals/swebench/run-instance.sh are reachable. docker build \ -f evals/swebench/Dockerfile.agent \ -t claude-mem/swebench-agent:latest \ diff --git a/evals/swebench/eval.sh b/evals/swebench/eval.sh index 6f7f9968..a6ef9b93 100755 --- a/evals/swebench/eval.sh +++ b/evals/swebench/eval.sh @@ -1,18 +1,6 @@ #!/usr/bin/env bash set -euo pipefail -# eval.sh — Thin wrapper around `python -m swebench.harness.run_evaluation`. -# -# Required env: -# RUN_ID Identifier for this evaluation run (matches predictions dir). -# Optional env: -# MAX_WORKERS Parallel worker count for the harness (default: 4). -# DATASET HF dataset name (default: princeton-nlp/SWE-bench_Verified). -# TIMEOUT Per-instance timeout in seconds (default: 1800). -# -# Reports land at: -# logs/run_evaluation/$RUN_ID/claude-opus-4-7+claude-mem//report.json - : "${RUN_ID:?RUN_ID is required (e.g. RUN_ID=smoke-001)}" MAX_WORKERS="${MAX_WORKERS:-4}" DATASET="${DATASET:-princeton-nlp/SWE-bench_Verified}" @@ -30,7 +18,6 @@ if [[ ! -f "$PREDICTIONS" ]]; then exit 1 fi -# Harness REQUIRES Docker — fail fast with a clean message if it's not running. if ! command -v docker >/dev/null 2>&1; then echo "ERROR: docker CLI not found on PATH. The SWE-bench harness requires Docker." >&2 exit 1 @@ -40,13 +27,11 @@ if ! docker info >/dev/null 2>&1; then exit 1 fi -# Create/reuse a dedicated venv so we don't pollute the system Python. VENV_DIR=".venv-swebench" if [[ ! -d "$VENV_DIR" ]]; then echo "[eval.sh] Creating Python venv at $VENV_DIR ..." python3 -m venv "$VENV_DIR" fi -# shellcheck disable=SC1091 source "$VENV_DIR/bin/activate" echo "[eval.sh] Installing/updating swebench in $VENV_DIR ..." diff --git a/evals/swebench/run-batch.py b/evals/swebench/run-batch.py index 87890473..47c8f3be 100755 --- a/evals/swebench/run-batch.py +++ b/evals/swebench/run-batch.py @@ -37,9 +37,6 @@ from typing import Any, Iterable from datasets import load_dataset - -# Hidden-from-agent fields per the plan. We MUST NOT pass these to the agent -# container — they are evaluator-only ground truth. HIDDEN_AGENT_FIELDS = ( "patch", "test_patch", @@ -49,7 +46,6 @@ HIDDEN_AGENT_FIELDS = ( "version", ) - def extract_oauth_credentials() -> Path | None: """ Extract Claude Code OAuth credentials (from a Max/Pro subscription) to a @@ -76,12 +72,8 @@ def extract_oauth_credentials() -> Path | None: ) temp_path = Path(temp.name) temp.close() - # Clean up on process exit, even on crash. atexit.register(lambda: temp_path.unlink(missing_ok=True)) - # macOS: try Keychain first (primary storage on Darwin). On miss, fall - # through to the on-disk credentials file — some macOS setups (older CLI, - # migrated machines) only have the file form. if platform.system() == "Darwin": try: completed = subprocess.run( @@ -100,15 +92,12 @@ def extract_oauth_credentials() -> Path | None: temp_path.write_text(completed.stdout.strip(), encoding="utf-8") temp_path.chmod(stat.S_IRUSR | stat.S_IWUSR) return temp_path - # else fall through to the on-disk credentials check below except FileNotFoundError: print( "WARN: `security` command not available; trying on-disk creds.", file=sys.stderr, ) - # fall through to the on-disk credentials check below - # Both platforms (and macOS fallback): read the on-disk credentials file. creds_file = Path.home() / ".claude" / ".credentials.json" if creds_file.exists(): temp_path.write_text(creds_file.read_text(encoding="utf-8"), encoding="utf-8") @@ -124,7 +113,6 @@ def extract_oauth_credentials() -> Path | None: ) return None - def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser( description="Run the claude-mem SWE-bench agent on a batch of instances.", @@ -199,7 +187,6 @@ def parse_args() -> argparse.Namespace: ) return parser.parse_args() - def select_instances( dataset: Iterable[dict[str, Any]], instance_ids: list[str] | None, @@ -221,7 +208,6 @@ def select_instances( rows = rows[:limit] return rows - def append_prediction_row( predictions_path: Path, instance_id: str, @@ -240,14 +226,12 @@ def append_prediction_row( with predictions_path.open("a", encoding="utf-8") as fp: fp.write(line) - def copy_log_if_exists(src: Path, dst: Path) -> None: """Copy a log file from the shared scratch volume into the run-log directory, if present.""" if src.exists() and src.is_file(): dst.parent.mkdir(parents=True, exist_ok=True) shutil.copy2(src, dst) - def run_one_instance( instance: dict[str, Any], image: str, @@ -278,7 +262,6 @@ def run_one_instance( instance_log_dir.mkdir(parents=True, exist_ok=True) stderr_log_path = instance_log_dir / "stderr.log" - # Per-instance scratch dir — MUST NOT be shared across containers. scratch_dir = Path(tempfile.mkdtemp(prefix=f"swebench-{instance_id}-")) problem_file = scratch_dir / "problem.txt" problem_file.write_text(problem_statement, encoding="utf-8") @@ -286,18 +269,9 @@ def run_one_instance( status: str = "failed" model_patch: str = "" - # Uniquely named so the TimeoutExpired handler can kill it without racing - # other instances on the host. container_name = f"swebench-agent-{instance_id}-{os.getpid()}-{threading.get_ident()}" try: - # The orchestrator owns JSONL writes under `predictions_lock` to avoid - # racy concurrent appends across containers — so we DO NOT mount the - # predictions directory into the container. Instead, the agent writes - # its authoritative diff to /scratch/model_patch.diff (via - # CLAUDE_MEM_OUTPUT_DIR), plus ingest/fix logs to the same dir. The - # 5th CLI arg to run-instance.sh is only used in standalone smoke-test - # mode; here we point it at a throwaway path inside the container. cmd: list[str] = [ "docker", "run", @@ -317,7 +291,6 @@ def run_one_instance( f"{oauth_creds_path}:/auth/.credentials.json:ro", ] else: - # Pay-per-call path. cmd += ["-e", "ANTHROPIC_API_KEY"] cmd += [ image, @@ -336,18 +309,12 @@ def run_one_instance( text=True, check=False, ) - # Persist stderr so post-mortem is possible even on success. stderr_log_path.write_text( f"=== STDOUT ===\n{completed.stdout}\n=== STDERR ===\n{completed.stderr}\n", encoding="utf-8", ) if completed.returncode == 0: - # Read the diff the agent wrote to the shared predictions volume. - # The container writes its own prediction line; we prefer to - # write our own authoritative row here from the diff file the - # agent left in /scratch. If the agent wrote a diff file, use - # it; otherwise fall back to empty patch. diff_file = scratch_dir / "model_patch.diff" if diff_file.exists(): diff_text = diff_file.read_text(encoding="utf-8") @@ -355,19 +322,14 @@ def run_one_instance( model_patch = diff_text status = "succeeded" else: - status = "failed" # empty diff + status = "failed" else: - # Container did not leave a diff file — treat as failure - # but still emit an empty-patch row below. status = "failed" else: status = "failed" except subprocess.TimeoutExpired as exc: status = "timed_out" - # subprocess.run killed the docker CLI, but the container may - # still be running. Force-remove it by name so we don't leak - # containers across the batch. subprocess.run( ["docker", "rm", "-f", container_name], capture_output=True, @@ -381,11 +343,9 @@ def run_one_instance( encoding="utf-8", ) - # Copy per-turn logs left by the agent in the shared scratch volume. copy_log_if_exists(scratch_dir / "ingest.jsonl", instance_log_dir / "ingest.jsonl") copy_log_if_exists(scratch_dir / "fix.jsonl", instance_log_dir / "fix.jsonl") - # Always write a row — never silently drop an instance. append_prediction_row( predictions_path=predictions_path, instance_id=instance_id, @@ -394,7 +354,7 @@ def run_one_instance( lock=predictions_lock, ) - except Exception as exc: # pragma: no cover — defensive + except Exception as exc: status = "failed" try: stderr_log_path.write_text( @@ -411,12 +371,10 @@ def run_one_instance( lock=predictions_lock, ) finally: - # Per-instance scratch must not leak across containers. shutil.rmtree(scratch_dir, ignore_errors=True) return status, instance_id - def main() -> int: args = parse_args() @@ -434,9 +392,8 @@ def main() -> int: ) predictions_dir = predictions_path.parent - run_dir = predictions_dir # logs land in evals/swebench/runs/// + run_dir = predictions_dir predictions_dir.mkdir(parents=True, exist_ok=True) - # Don't silently discard partial results from a prior run. if predictions_path.exists() and predictions_path.stat().st_size > 0: if not args.overwrite: print( @@ -451,7 +408,6 @@ def main() -> int: ) predictions_path.write_text("", encoding="utf-8") - # Resolve auth: OAuth (Max/Pro subscription) or API key. oauth_creds_path: Path | None = None if args.auth in ("oauth", "auto"): oauth_creds_path = extract_oauth_credentials() @@ -488,10 +444,6 @@ def main() -> int: print("No instances selected; nothing to do.", file=sys.stderr) return 0 - # Scrub hidden-from-agent fields defensively. The agent container only - # receives instance_id/repo/base_commit/problem_statement via CLI args + - # the per-instance problem file — the hidden fields never leave this - # process. This loop makes that invariant explicit. for row in instances: for key in HIDDEN_AGENT_FIELDS: row.pop(key, None) @@ -530,12 +482,23 @@ def main() -> int: instance_id = future_to_id[future] try: status, _ = future.result() - except Exception as exc: # pragma: no cover — defensive + except Exception as exc: status = "failed" print( f"[{instance_id}] orchestrator future raised: {exc!r}", file=sys.stderr, ) + # The orchestrator died before run_one_instance could write a + # row. Append a fallback so this instance still appears in + # predictions.jsonl — preserving the "never drop an instance" + # guarantee that downstream evaluation depends on. + append_prediction_row( + predictions_path=predictions_path, + instance_id=instance_id, + model_patch="", + model_name_or_path=model_name_or_path, + lock=predictions_lock, + ) if status == "succeeded": succeeded += 1 @@ -553,9 +516,7 @@ def main() -> int: print( f"{total} total, {succeeded} succeeded, {failed} failed, {timed_out} timed out", ) - # Per plan: exit 0 even if some instances failed. return 0 - if __name__ == "__main__": sys.exit(main()) diff --git a/evals/swebench/run-instance.sh b/evals/swebench/run-instance.sh index b87797b4..eaa41000 100755 --- a/evals/swebench/run-instance.sh +++ b/evals/swebench/run-instance.sh @@ -1,16 +1,6 @@ #!/usr/bin/env bash set -euo pipefail -# run-instance.sh — runs Claude Code + claude-mem against a single SWE-bench -# instance using the two-turn protocol (ingest, then fix), and appends a -# prediction JSONL row to OUT_PREDICTIONS_PATH. -# -# Usage: -# run-instance.sh INSTANCE_ID REPO_SLUG BASE_COMMIT PROBLEM_STATEMENT_FILE OUT_PREDICTIONS_PATH -# -# Required env: -# ANTHROPIC_API_KEY - if [[ $# -ne 5 ]]; then echo "Usage: $0 INSTANCE_ID REPO_SLUG BASE_COMMIT PROBLEM_STATEMENT_FILE OUT_PREDICTIONS_PATH" >&2 exit 2 @@ -22,12 +12,6 @@ BASE_COMMIT="$3" PROBLEM_STATEMENT_FILE="$4" OUT_PREDICTIONS_PATH="$5" -# Auth: either ANTHROPIC_API_KEY (pay-per-call) OR a pre-extracted OAuth -# credentials file from a Claude Max/Pro subscription (flat-fee, but subject -# to Anthropic's usage limits — batch-scale runs may exhaust the 5h window). -# run-batch.py extracts OAuth creds from host Keychain/file and mounts them -# at CLAUDE_MEM_CREDENTIALS_FILE; standalone smoke-test can do the same, or -# set ANTHROPIC_API_KEY directly. if [[ -z "${ANTHROPIC_API_KEY:-}" && -z "${CLAUDE_MEM_CREDENTIALS_FILE:-}" ]]; then echo "ERROR: one of ANTHROPIC_API_KEY or CLAUDE_MEM_CREDENTIALS_FILE is required" >&2 exit 1 @@ -45,33 +29,20 @@ fi MODEL_NAME="claude-opus-4-7+claude-mem" -# Per-instance ephemeral scratch dir — isolates ~/.claude/ and ~/.claude-mem/. SCRATCH=$(mktemp -d) REPO_DIR="$SCRATCH/repo" MEM_DIR="$SCRATCH/.claude-mem" CLAUDE_DIR="$SCRATCH/.claude" mkdir -p "$MEM_DIR" "$CLAUDE_DIR" -# If using OAuth, seed the isolated CLAUDE_DIR with the mounted credentials -# file so Claude Code finds them at HOME=$SCRATCH → ~/.claude/.credentials.json. -# chmod 600 to match what `claude login` writes (it checks permissions). if [[ -n "${CLAUDE_MEM_CREDENTIALS_FILE:-}" ]]; then cp "$CLAUDE_MEM_CREDENTIALS_FILE" "$CLAUDE_DIR/.credentials.json" chmod 600 "$CLAUDE_DIR/.credentials.json" fi -# Directory where artifacts the batch orchestrator reads (model_patch.diff, -# ingest.jsonl, fix.jsonl) are written. When run via `docker run -v -# :/scratch` from run-batch.py, the orchestrator sets -# CLAUDE_MEM_OUTPUT_DIR=/scratch so these files are visible on the host. In -# standalone/smoke-test mode the default keeps artifacts in the ephemeral -# scratch dir alongside the repo. OUTPUT_DIR="${CLAUDE_MEM_OUTPUT_DIR:-$SCRATCH}" mkdir -p "$OUTPUT_DIR" -# Always write a prediction row (even on failure) so batch mode stays aligned. -# The trap emits an empty-patch row if we exit before the success path sets -# PREDICTION_EMITTED=1, then cleans up SCRATCH. DIFF_OUT="$OUTPUT_DIR/model_patch.diff" INGEST_LOG="$OUTPUT_DIR/ingest.jsonl" FIX_LOG="$OUTPUT_DIR/fix.jsonl" @@ -80,7 +51,6 @@ PREDICTION_EMITTED=0 cleanup() { local exit_code=$? if [[ "$PREDICTION_EMITTED" -ne 1 ]]; then - # Ensure the orchestrator sees an (empty) diff file even on early exit. : > "$DIFF_OUT" 2>/dev/null || true jq -nc \ --arg id "$INSTANCE_ID" \ @@ -94,11 +64,6 @@ cleanup() { } trap cleanup EXIT -# Shallow clone + fetch the exact commit. Saves minutes on large repos -# (sympy/django/scikit-learn) vs. a full-history clone. Fallback to a full -# clone if the server rejects the by-commit fetch (GitHub supports -# uploadpack.allowReachableSHA1InWant by default on public repos, but mirrors -# may not). if ! { git clone --depth 1 --no-single-branch "https://github.com/${REPO_SLUG}.git" "$REPO_DIR" \ && git -C "$REPO_DIR" fetch --depth 1 origin "$BASE_COMMIT"; }; then echo "WARN: shallow fetch failed; falling back to full clone" >&2 @@ -107,7 +72,6 @@ if ! { git clone --depth 1 --no-single-branch "https://github.com/${REPO_SLUG}.g fi git -C "$REPO_DIR" reset --hard "$BASE_COMMIT" -# ---------- Turn 1: Ingest (populate memory via PostToolUse hook) ---------- INGEST_PROMPT="Please learn about the codebase by systematically and thoroughly reading EVERY SOURCE FILE IN FULL, no matter how many there are. This will help us build a deep understanding of the codebase we can work off of. Don't worry about cost. This is critical and non-negotiable." SESSION_ID=$(uuidgen | tr '[:upper:]' '[:lower:]') @@ -131,7 +95,6 @@ if [[ "$INGEST_EXIT" -ne 0 ]]; then echo "WARN: ingest turn exited with $INGEST_EXIT; continuing to fix turn" >&2 fi -# ---------- Turn 2: Fix (consume memory via mem-search slash command) ---------- PROBLEM=$(cat "$PROBLEM_STATEMENT_FILE") QUERY=$(printf '%s' "$PROBLEM" | tr -s '[:space:]' ' ' | cut -c1-200) @@ -161,9 +124,6 @@ if [[ "$FIX_EXIT" -ne 0 ]]; then echo "WARN: fix turn exited with $FIX_EXIT; will still emit prediction row" >&2 fi -# ---------- Capture diff and emit prediction row ---------- -# Write the diff to DIFF_OUT first (authoritative for the batch orchestrator), -# then read it back for the JSONL row (kept for standalone/smoke-test use). git -C "$REPO_DIR" diff > "$DIFF_OUT" || : > "$DIFF_OUT" DIFF=$(cat "$DIFF_OUT") diff --git a/evals/swebench/smoke-test.sh b/evals/swebench/smoke-test.sh index a43cd744..81b10d31 100755 --- a/evals/swebench/smoke-test.sh +++ b/evals/swebench/smoke-test.sh @@ -1,20 +1,6 @@ #!/usr/bin/env bash set -euo pipefail -# smoke-test.sh — runs ONE SWE-bench instance end-to-end against the agent -# container using OAuth credentials extracted from the host. Use this to -# verify the two-turn protocol + /claude-mem:mem-search slash resolution -# before kicking off a batch run. -# -# Usage: -# evals/swebench/smoke-test.sh [INSTANCE_ID] -# -# Defaults to sympy__sympy-24152 (an easy Verified instance) if no arg given. -# -# Outputs: -# evals/swebench/runs/smoke//{ingest.jsonl,fix.jsonl,model_patch.diff} -# evals/swebench/runs/smoke/predictions.jsonl - INSTANCE_ID="${1:-sympy__sympy-24152}" DATASET="${DATASET:-princeton-nlp/SWE-bench_Lite}" IMAGE="${IMAGE:-claude-mem/swebench-agent:latest}" @@ -26,12 +12,9 @@ RUN_DIR="$REPO_ROOT/evals/swebench/runs/smoke/$INSTANCE_ID" PREDICTIONS="$REPO_ROOT/evals/swebench/runs/smoke/predictions.jsonl" mkdir -p "$RUN_DIR" "$(dirname "$PREDICTIONS")" -# --- Extract OAuth credentials --- CREDS_FILE="$(mktemp -t claude-mem-creds.XXXXXX.json)" trap 'rm -f "$CREDS_FILE"' EXIT -# Try macOS Keychain first (primary on Darwin), then fall through to the -# on-disk credentials file — matches docker/claude-mem/run.sh behavior. creds_obtained=0 if [[ "$(uname)" == "Darwin" ]]; then if security find-generic-password -s 'Claude Code-credentials' -w > "$CREDS_FILE" 2>/dev/null \ @@ -49,7 +32,6 @@ if [[ "$creds_obtained" -eq 0 ]]; then fi chmod 600 "$CREDS_FILE" -# --- Fetch instance data from HuggingFace via a small Python helper --- INSTANCE_JSON="$(mktemp)" trap 'rm -f "$CREDS_FILE" "$INSTANCE_JSON"' EXIT python3 - "$INSTANCE_ID" "$DATASET" > "$INSTANCE_JSON" <<'PY' @@ -75,10 +57,6 @@ PY SCRATCH="$(mktemp -d -t claude-mem-smoke.XXXXXX)" trap 'rm -f "$CREDS_FILE" "$INSTANCE_JSON"; rm -rf "$SCRATCH"' EXIT -# Parse the instance JSON once: print repo + base_commit to stdout, write the -# problem statement directly to $SCRATCH/problem.txt. INSTANCE_JSON is passed -# as argv so stdin is free for the `python3 -` heredoc script body (previously -# both were competing for stdin, which made json.load see the heredoc's EOF). read -r REPO BASE_COMMIT < <( python3 - "$SCRATCH" "$INSTANCE_JSON" <<'PY' import json, os, sys @@ -94,9 +72,6 @@ echo "=== Running $INSTANCE_ID ($REPO @ $BASE_COMMIT) ===" >&2 echo "Scratch: $SCRATCH" >&2 echo "Logs will land in: $RUN_DIR" >&2 -# Pick a wall-clock timeout binary. Linux ships `timeout`; macOS needs -# `gtimeout` from coreutils (brew install coreutils). If neither is available, -# warn and run without a cap — the smoke test is manual anyway. TIMEOUT_CMD=() if command -v timeout >/dev/null 2>&1; then TIMEOUT_CMD=(timeout "$TIMEOUT") @@ -106,8 +81,6 @@ else echo "WARN: no \`timeout\`/\`gtimeout\` on PATH; container runs uncapped" >&2 fi -# Name the container so we can force-remove it if the wall-clock timeout -# fires (SIGTERM from timeout leaves the container state open briefly). CONTAINER_NAME="claude-mem-smoke-$INSTANCE_ID-$$" set +e @@ -123,18 +96,14 @@ DOCKER_EXIT=$? set -e if [[ "$DOCKER_EXIT" -eq 124 ]]; then - # `timeout` signals TERM and returns 124 on timeout. Force-remove the - # container in case docker hasn't reaped it yet. echo "ERROR: docker run exceeded ${TIMEOUT}s wall-clock; removing container" >&2 docker rm -f "$CONTAINER_NAME" >/dev/null 2>&1 || true fi -# Copy artifacts from scratch → RUN_DIR for f in ingest.jsonl fix.jsonl model_patch.diff; do [[ -f "$SCRATCH/$f" ]] && cp "$SCRATCH/$f" "$RUN_DIR/$f" done -# Emit authoritative prediction row DIFF_FILE="$SCRATCH/model_patch.diff" DIFF="" [[ -f "$DIFF_FILE" ]] && DIFF="$(cat "$DIFF_FILE")" diff --git a/evals/swebench/summarize.py b/evals/swebench/summarize.py index f14388bb..34fdf6dd 100755 --- a/evals/swebench/summarize.py +++ b/evals/swebench/summarize.py @@ -10,7 +10,6 @@ import json import sys from pathlib import Path - def load_expected_instance_ids(predictions_path: Path) -> list[str]: """Read instance_ids from a predictions.jsonl file (one JSON object per line).""" instance_ids: list[str] = [] @@ -38,7 +37,6 @@ def load_expected_instance_ids(predictions_path: Path) -> list[str]: instance_ids.append(instance_id) return instance_ids - def load_run_results( run_id: str, model_name: str, @@ -81,8 +79,6 @@ def load_run_results( error_count += 1 continue - # SWE-bench harness typically nests per-instance data under the - # instance_id key; fall back to the top-level dict for flexibility. inner = report_data.get(instance_id, report_data) resolved_value = inner.get("resolved") if resolved_value is True: @@ -116,7 +112,6 @@ def load_run_results( "error_count": error_count, } - def format_resolved_cell(resolved: bool | None) -> str: if resolved is True: return "yes" @@ -124,7 +119,6 @@ def format_resolved_cell(resolved: bool | None) -> str: return "no" return "error" - def render_summary_markdown(run_id: str, results: dict) -> str: total = ( results["resolved_count"] @@ -147,13 +141,11 @@ def render_summary_markdown(run_id: str, results: dict) -> str: for instance_id, record in results["per_instance"].items(): resolved_cell = format_resolved_cell(record["resolved"]) notes_cell = record.get("notes", "") or "" - # Escape pipe chars in notes to avoid breaking markdown tables. notes_cell = notes_cell.replace("|", "\\|") lines.append(f"| {instance_id} | {resolved_cell} | {notes_cell} |") lines.append("") return "\n".join(lines) - def render_diff_markdown( current_run_id: str, other_run_id: str, @@ -214,7 +206,6 @@ def render_diff_markdown( lines.append("") return "\n".join(lines) - def main() -> int: parser = argparse.ArgumentParser( description="Summarize SWE-bench evaluation run results." @@ -242,7 +233,6 @@ def main() -> int: ) args = parser.parse_args() - # Resolve repo root from this script's location: evals/swebench/summarize.py script_path = Path(__file__).resolve() repo_root = script_path.parent.parent.parent @@ -303,6 +293,5 @@ def main() -> int: print(str(output_path)) return 0 - if __name__ == "__main__": raise SystemExit(main()) diff --git a/install/public/install.sh b/install/public/install.sh index d3be8cfa..ce02c2da 100644 --- a/install/public/install.sh +++ b/install/public/install.sh @@ -1,16 +1,11 @@ #!/bin/bash set -euo pipefail -# claude-mem installer redirect -# The old curl-pipe-bash installer has been replaced by npx claude-mem. -# This script now redirects users to the new install method. - -# Colors RED='\033[0;31m' GREEN='\033[0;32m' CYAN='\033[0;36m' YELLOW='\033[0;33m' -NC='\033[0m' # No Color +NC='\033[0m' echo "" echo -e "${YELLOW}The curl-pipe-bash installer has been replaced.${NC}" @@ -19,7 +14,7 @@ echo -e "${GREEN}Install claude-mem with a single command:${NC}" echo "" echo -e " ${CYAN}npx claude-mem install${NC}" echo "" -echo -e "This requires Node.js >= 18. Get it from ${CYAN}https://nodejs.org${NC}" +echo -e "This requires Node.js >= 20. Get it from ${CYAN}https://nodejs.org${NC}" echo "" echo -e "For more info, visit: ${CYAN}https://docs.claude-mem.ai/installation${NC}" echo "" diff --git a/install/public/installer.js b/install/public/installer.js index 3f31d299..85129e9a 100644 --- a/install/public/installer.js +++ b/install/public/installer.js @@ -1,9 +1,5 @@ #!/usr/bin/env node -// claude-mem installer redirect -// The old bundled installer has been replaced by npx claude-mem. -// This script now redirects users to the new install method. - console.log(''); console.log('\x1b[33mThe bundled installer has been replaced.\x1b[0m'); console.log(''); diff --git a/openclaw/Dockerfile.e2e b/openclaw/Dockerfile.e2e index fcc0bcbd..dff9532d 100644 --- a/openclaw/Dockerfile.e2e +++ b/openclaw/Dockerfile.e2e @@ -1,35 +1,18 @@ -# Dockerfile.e2e — End-to-end test: install claude-mem plugin on a real OpenClaw instance -# Simulates the complete plugin installation flow a user would follow. -# -# Usage: -# docker build -f Dockerfile.e2e -t openclaw-e2e-test . && docker run --rm openclaw-e2e-test -# -# Interactive (for human testing): -# docker run --rm -it openclaw-e2e-test /bin/bash FROM ghcr.io/openclaw/openclaw:main USER root -# Install curl for health checks in e2e-verify.sh, and TypeScript for building RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/* RUN npm install -g typescript@5 -# Create staging directory for the plugin source WORKDIR /tmp/claude-mem-plugin -# Copy plugin source files COPY package.json tsconfig.json openclaw.plugin.json ./ COPY src/ ./src/ -# Build the plugin (TypeScript → JavaScript) -# NODE_ENV=production is set in the base image; override to install devDependencies RUN NODE_ENV=development npm install && npx tsc -# Create the installable plugin package: -# OpenClaw `plugins install` expects package.json with openclaw.extensions field. -# The package name must match the plugin ID in openclaw.plugin.json (claude-mem). -# Only include the main plugin entry point, not test/mock files. RUN mkdir -p /tmp/claude-mem-installable/dist && \ cp dist/index.js /tmp/claude-mem-installable/dist/ && \ cp dist/index.d.ts /tmp/claude-mem-installable/dist/ 2>/dev/null || true && \ @@ -45,25 +28,19 @@ RUN mkdir -p /tmp/claude-mem-installable/dist && \ require('fs').writeFileSync('/tmp/claude-mem-installable/package.json', JSON.stringify(pkg, null, 2)); \ " -# Switch back to app directory and node user for installation WORKDIR /app USER node -# Create the OpenClaw config directory RUN mkdir -p /home/node/.openclaw -# Install the plugin using OpenClaw's official CLI RUN node openclaw.mjs plugins install /tmp/claude-mem-installable -# Enable the plugin RUN node openclaw.mjs plugins enable claude-mem -# Copy the e2e verification script and mock worker COPY --chown=node:node e2e-verify.sh /app/e2e-verify.sh USER root RUN chmod +x /app/e2e-verify.sh && \ cp /tmp/claude-mem-plugin/dist/mock-worker.js /app/mock-worker.js USER node -# Default: run the automated verification CMD ["/bin/bash", "/app/e2e-verify.sh"] diff --git a/openclaw/e2e-verify.sh b/openclaw/e2e-verify.sh index 464e08e6..7ae97185 100755 --- a/openclaw/e2e-verify.sh +++ b/openclaw/e2e-verify.sh @@ -1,13 +1,4 @@ #!/usr/bin/env bash -# e2e-verify.sh — Automated E2E verification for claude-mem plugin on OpenClaw -# -# This script verifies the complete plugin installation and operation flow: -# 1. Plugin is installed and visible in OpenClaw -# 2. Plugin loads correctly when gateway starts -# 3. Mock worker SSE stream is consumed by the plugin -# 4. Observations are received and formatted -# -# Exit 0 = all checks passed, Exit 1 = failure set -euo pipefail @@ -32,11 +23,8 @@ section() { echo "=== $1 ===" } -# ─── Phase 1: Plugin Discovery ─── - section "Phase 1: Plugin Discovery" -# Check plugin is listed PLUGIN_LIST=$(node /app/openclaw.mjs plugins list 2>&1) if echo "$PLUGIN_LIST" | grep -q "claude-mem"; then pass "Plugin appears in 'plugins list'" @@ -45,7 +33,6 @@ else echo "$PLUGIN_LIST" fi -# Check plugin info PLUGIN_INFO=$(node /app/openclaw.mjs plugins info claude-mem 2>&1 || true) if echo "$PLUGIN_INFO" | grep -qi "claude-mem"; then pass "Plugin info shows claude-mem details" @@ -54,11 +41,9 @@ else echo "$PLUGIN_INFO" fi -# Check plugin is enabled if echo "$PLUGIN_LIST" | grep -A1 "claude-mem" | grep -qi "enabled\|loaded"; then pass "Plugin is enabled" else - # Try to check via info if echo "$PLUGIN_INFO" | grep -qi "enabled\|loaded"; then pass "Plugin is enabled (via info)" else @@ -67,7 +52,6 @@ else fi fi -# Check plugin doctor reports no issues DOCTOR_OUT=$(node /app/openclaw.mjs plugins doctor 2>&1 || true) if echo "$DOCTOR_OUT" | grep -qi "no.*issue\|0 issue"; then pass "Plugin doctor reports no issues" @@ -76,17 +60,12 @@ else echo "$DOCTOR_OUT" fi -# ─── Phase 2: Plugin Files ─── - section "Phase 2: Plugin Files" -# Check extension directory exists EXTENSIONS_DIR="/home/node/.openclaw/extensions/openclaw-plugin" if [ ! -d "$EXTENSIONS_DIR" ]; then - # Try alternative naming EXTENSIONS_DIR="/home/node/.openclaw/extensions/claude-mem" if [ ! -d "$EXTENSIONS_DIR" ]; then - # Search for it FOUND_DIR=$(find /home/node/.openclaw/extensions/ -name "openclaw.plugin.json" -exec dirname {} \; 2>/dev/null | head -1 || true) if [ -n "$FOUND_DIR" ]; then EXTENSIONS_DIR="$FOUND_DIR" @@ -101,7 +80,6 @@ else ls -la /home/node/.openclaw/extensions/ 2>/dev/null || echo " (extensions dir not found)" fi -# Check key files exist for FILE in "openclaw.plugin.json" "dist/index.js" "package.json"; do if [ -f "$EXTENSIONS_DIR/$FILE" ]; then pass "File exists: $FILE" @@ -110,16 +88,12 @@ for FILE in "openclaw.plugin.json" "dist/index.js" "package.json"; do fi done -# ─── Phase 3: Mock Worker + Plugin Integration ─── - section "Phase 3: Mock Worker + Plugin Integration" -# Start mock worker in background echo " Starting mock claude-mem worker..." node /app/mock-worker.js & MOCK_PID=$! -# Wait for mock worker to be ready for i in $(seq 1 10); do if curl -sf http://localhost:37777/health > /dev/null 2>&1; then break @@ -134,7 +108,6 @@ else kill $MOCK_PID 2>/dev/null || true fi -# Test SSE stream connectivity (curl with max-time to capture initial SSE frame) SSE_TEST=$(curl -s --max-time 2 http://localhost:37777/stream 2>/dev/null || true) if echo "$SSE_TEST" | grep -q "connected"; then pass "SSE stream returns connected event" @@ -143,13 +116,8 @@ else echo " Got: $(echo "$SSE_TEST" | head -5)" fi -# ─── Phase 4: Gateway + Plugin Load ─── - section "Phase 4: Gateway Startup with Plugin" -# Create a minimal config that enables the plugin with the mock worker. -# The memory slot must be set to "claude-mem" to match what `plugins install` configured. -# Gateway auth is disabled via token for headless testing. mkdir -p /home/node/.openclaw cat > /home/node/.openclaw/openclaw.json << 'EOFCONFIG' { @@ -183,16 +151,13 @@ EOFCONFIG pass "OpenClaw config written with plugin enabled" -# Start gateway in background and capture output GATEWAY_LOG="/tmp/gateway.log" echo " Starting OpenClaw gateway (timeout 15s)..." OPENCLAW_GATEWAY_TOKEN=e2e-test-token timeout 15 node /app/openclaw.mjs gateway --allow-unconfigured --verbose --token e2e-test-token > "$GATEWAY_LOG" 2>&1 & GATEWAY_PID=$! -# Give the gateway time to start and load plugins sleep 5 -# Check if gateway started if kill -0 $GATEWAY_PID 2>/dev/null; then pass "Gateway process is running" else @@ -201,7 +166,6 @@ else cat "$GATEWAY_LOG" 2>/dev/null | tail -30 fi -# Check gateway log for plugin load messages if grep -qi "claude-mem" "$GATEWAY_LOG" 2>/dev/null; then pass "Gateway log mentions claude-mem plugin" else @@ -210,29 +174,24 @@ else tail -20 "$GATEWAY_LOG" 2>/dev/null fi -# Check for plugin loaded message if grep -q "plugin loaded" "$GATEWAY_LOG" 2>/dev/null || grep -q "v1.0.0" "$GATEWAY_LOG" 2>/dev/null; then pass "Plugin load message found in gateway log" else fail "Plugin load message not found" fi -# Check for observation feed messages if grep -qi "observation feed" "$GATEWAY_LOG" 2>/dev/null; then pass "Observation feed activity in gateway log" else fail "No observation feed activity detected" fi -# Check for SSE connection to mock worker if grep -qi "connected.*SSE\|SSE.*stream\|connecting.*SSE" "$GATEWAY_LOG" 2>/dev/null; then pass "SSE connection activity detected" else fail "No SSE connection activity in log" fi -# ─── Cleanup ─── - section "Cleanup" kill $GATEWAY_PID 2>/dev/null || true kill $MOCK_PID 2>/dev/null || true @@ -240,8 +199,6 @@ wait $GATEWAY_PID 2>/dev/null || true wait $MOCK_PID 2>/dev/null || true echo " Processes stopped." -# ─── Summary ─── - echo "" echo "===============================" echo " E2E Test Results" diff --git a/openclaw/install.sh b/openclaw/install.sh index a348eb99..f40d09b8 100755 --- a/openclaw/install.sh +++ b/openclaw/install.sh @@ -1,27 +1,9 @@ #!/usr/bin/env bash set -euo pipefail -# claude-mem OpenClaw Plugin Installer -# Installs the claude-mem persistent memory plugin for OpenClaw gateways. -# -# Usage: -# curl -fsSL https://install.cmem.ai/openclaw.sh | bash -# # Or with options: -# curl -fsSL https://install.cmem.ai/openclaw.sh | bash -s -- --provider=gemini --api-key=YOUR_KEY -# # Direct execution: -# bash install.sh [--non-interactive] [--upgrade] [--provider=claude|gemini|openrouter] [--api-key=KEY] - -############################################################################### -# Constants -############################################################################### - readonly MIN_BUN_VERSION="1.1.14" readonly INSTALLER_VERSION="1.0.0" -############################################################################### -# Argument parsing -############################################################################### - NON_INTERACTIVE="" CLI_PROVIDER="" CLI_API_KEY="" @@ -68,37 +50,23 @@ while [[ $# -gt 0 ]]; do esac done -############################################################################### -# TTY detection — ensure interactive prompts work under curl | bash -# When piped, stdin reads from curl's output, not the terminal. -# We open /dev/tty on fd 3 and read interactive input from there. -############################################################################### - TTY_FD=0 setup_tty() { if [[ -t 0 ]]; then - # stdin IS a terminal — use it directly TTY_FD=0 elif [[ "$NON_INTERACTIVE" == "true" ]]; then - # In non-interactive mode, do not require /dev/tty TTY_FD=0 elif [[ -r /dev/tty ]]; then - # stdin is piped (curl | bash) but /dev/tty is available and readable exec 3&2 echo "Use --non-interactive or run directly: bash install.sh" >&2 exit 1 fi } -############################################################################### -# Color utilities — auto-detect terminal color support -############################################################################### - if [[ -t 1 ]] && [[ "${TERM:-}" != "dumb" ]]; then readonly COLOR_RED='\033[0;31m' readonly COLOR_GREEN='\033[0;32m' @@ -132,17 +100,10 @@ prompt_user() { echo -en "${COLOR_CYAN}?${COLOR_RESET} $* " } -# Read a line from the terminal (works even when stdin is piped from curl) -# Callers always pass -r via $@; shellcheck can't see through the delegation read_tty() { - # shellcheck disable=SC2162 read "$@" <&"$TTY_FD" } -############################################################################### -# Global cleanup trap — removes temp directories on unexpected exit -############################################################################### - CLEANUP_DIRS=() register_cleanup_dir() { @@ -166,10 +127,6 @@ cleanup_on_exit() { trap cleanup_on_exit EXIT -############################################################################### -# Prerequisite checks -############################################################################### - check_git() { if command -v git &>/dev/null; then return 0 @@ -196,24 +153,17 @@ check_git() { exit 1 } -############################################################################### -# Port conflict detection — check if port 37777 is already in use -############################################################################### - check_port_37777() { local port_in_use="" - # Try lsof first (macOS/Linux) if command -v lsof &>/dev/null; then if lsof -i :37777 -sTCP:LISTEN &>/dev/null; then port_in_use="true" fi - # Fallback to ss (Linux) elif command -v ss &>/dev/null; then if ss -tlnp 2>/dev/null | grep -q ':37777 '; then port_in_use="true" fi - # Fallback to curl probe elif command -v curl &>/dev/null; then local response response="$(curl -s -o /dev/null -w "%{http_code}" "http://127.0.0.1:37777/api/health" 2>/dev/null)" || true @@ -223,36 +173,23 @@ check_port_37777() { fi if [[ "$port_in_use" == "true" ]]; then - return 0 # port IS in use + return 0 fi - return 1 # port is free + return 1 } -############################################################################### -# Upgrade detection — check if claude-mem is already installed -############################################################################### - is_claude_mem_installed() { - # Check if the plugin directory exists with the worker script if find_claude_mem_install_dir 2>/dev/null; then return 0 fi return 1 } -############################################################################### -# JSON manipulation helper — jq with python3/node fallback -# Usage: ensure_jq_or_fallback [jq_args...] -# For simple read operations, returns the result on stdout. -# For write operations, updates the file in-place. -############################################################################### - ensure_jq_or_fallback() { local json_file="$1" shift local jq_filter="$1" shift - # remaining args are passed as jq --arg pairs if command -v jq &>/dev/null; then local tmp_file @@ -262,29 +199,16 @@ ensure_jq_or_fallback() { fi if command -v python3 &>/dev/null; then - # For complex jq filters, fall back to node instead - # Python is used only for simple operations : fi - # Fallback to node (always available — it's a dependency) - # This is a passthrough; callers that need node-specific logic - # should use node -e directly. This function is for jq compatibility. warn "jq not found — using node for JSON manipulation" return 1 } -############################################################################### -# Parse /api/health JSON response — extract worker metadata into globals -# Uses jq → python3 → node fallback chain (matching installer conventions) -# Sets: WORKER_VERSION, WORKER_AI_PROVIDER, WORKER_AI_AUTH_METHOD, -# WORKER_INITIALIZED, WORKER_REPORTED_PID, WORKER_UPTIME -############################################################################### - parse_health_json() { local raw_json="$1" - # Reset all health globals before parsing WORKER_VERSION="" WORKER_AI_PROVIDER="" WORKER_AI_AUTH_METHOD="" @@ -296,7 +220,6 @@ parse_health_json() { return 0 fi - # Try jq first (fastest, most reliable) if command -v jq &>/dev/null; then WORKER_VERSION="$(echo "$raw_json" | jq -r '.version // empty' 2>/dev/null)" || true WORKER_AI_PROVIDER="$(echo "$raw_json" | jq -r '.ai.provider // empty' 2>/dev/null)" || true @@ -307,7 +230,6 @@ parse_health_json() { return 0 fi - # Try python3 fallback if command -v python3 &>/dev/null; then local parsed parsed="$(INSTALLER_HEALTH_JSON="$raw_json" python3 -c " @@ -337,7 +259,6 @@ except Exception: WORKER_INITIALIZED="${health_fields[3]:-}" WORKER_REPORTED_PID="${health_fields[4]:-}" WORKER_UPTIME="${health_fields[5]:-}" - # Normalize python's None/empty representations [[ "$WORKER_VERSION" == "None" ]] && WORKER_VERSION="" [[ "$WORKER_AI_PROVIDER" == "None" ]] && WORKER_AI_PROVIDER="" [[ "$WORKER_AI_AUTH_METHOD" == "None" ]] && WORKER_AI_AUTH_METHOD="" @@ -348,7 +269,6 @@ except Exception: return 0 fi - # Fallback to node (always available — it's a dependency) local parsed parsed="$(INSTALLER_HEALTH_JSON="$raw_json" node -e " try { @@ -380,10 +300,6 @@ except Exception: fi } -############################################################################### -# Format uptime from milliseconds to human-readable (e.g., "2m 15s", "1h 23m") -############################################################################### - format_uptime_ms() { local ms="$1" local secs=$((ms / 1000)) @@ -396,10 +312,6 @@ format_uptime_ms() { fi } -############################################################################### -# Banner -############################################################################### - print_banner() { echo -e "${COLOR_MAGENTA}${COLOR_BOLD}" cat << 'BANNER' @@ -413,10 +325,6 @@ BANNER echo "" } -############################################################################### -# Platform detection -############################################################################### - PLATFORM="" IS_WSL="" @@ -448,10 +356,6 @@ detect_platform() { info "Detected platform: ${PLATFORM}${IS_WSL:+ (WSL)}" } -############################################################################### -# Version comparison — returns 0 if $1 >= $2 -############################################################################### - version_gte() { local v1="$1" v2="$2" local -a parts1 parts2 @@ -467,21 +371,14 @@ version_gte() { return 0 } -############################################################################### -# Bun detection and installation -# Translated from plugin/scripts/smart-install.js patterns -############################################################################### - BUN_PATH="" find_bun_path() { - # Try PATH first if command -v bun &>/dev/null; then BUN_PATH="$(command -v bun)" return 0 fi - # Check common installation paths (handles fresh installs before PATH reload) local -a bun_paths=( "${HOME}/.bun/bin/bun" "/usr/local/bin/bun" @@ -504,7 +401,6 @@ check_bun() { return 1 fi - # Verify minimum version local bun_version bun_version="$("$BUN_PATH" --version 2>/dev/null)" || return 1 @@ -529,7 +425,6 @@ install_bun() { exit 1 fi - # Re-detect after install (installer may have placed it in ~/.bun/bin) if ! find_bun_path; then error "Bun installation completed but binary not found in expected locations" error "Please restart your terminal and re-run this installer." @@ -541,21 +436,14 @@ install_bun() { success "Bun ${bun_version} installed at ${BUN_PATH}" } -############################################################################### -# uv detection and installation -# Translated from plugin/scripts/smart-install.js patterns -############################################################################### - UV_PATH="" find_uv_path() { - # Try PATH first if command -v uv &>/dev/null; then UV_PATH="$(command -v uv)" return 0 fi - # Check common installation paths (handles fresh installs before PATH reload) local -a uv_paths=( "${HOME}/.local/bin/uv" "${HOME}/.cargo/bin/uv" @@ -597,7 +485,6 @@ install_uv() { exit 1 fi - # Re-detect after install if ! find_uv_path; then error "uv installation completed but binary not found in expected locations" error "Please restart your terminal and re-run this installer." @@ -609,14 +496,9 @@ install_uv() { success "uv ${uv_version} installed at ${UV_PATH}" } -############################################################################### -# OpenClaw gateway detection -############################################################################### - OPENCLAW_PATH="" find_openclaw() { - # Try PATH first — check both "openclaw" and "openclaw.mjs" binary names for bin_name in openclaw openclaw.mjs; do if command -v "$bin_name" &>/dev/null; then OPENCLAW_PATH="$(command -v "$bin_name")" @@ -624,7 +506,6 @@ find_openclaw() { fi done - # Check common installation paths local -a openclaw_paths=( "${HOME}/.openclaw/openclaw.mjs" "/usr/local/bin/openclaw.mjs" @@ -634,7 +515,6 @@ find_openclaw() { "${HOME}/.npm-global/bin/openclaw" ) - # Also check for node_modules in common project locations if [[ -n "${NODE_PATH:-}" ]]; then openclaw_paths+=("${NODE_PATH}/openclaw/openclaw.mjs") fi @@ -667,7 +547,6 @@ check_openclaw() { success "OpenClaw gateway found at ${OPENCLAW_PATH}" } -# Run openclaw command — uses node for .mjs files, direct execution otherwise run_openclaw() { if [[ "$OPENCLAW_PATH" == *.mjs ]]; then node "$OPENCLAW_PATH" "$@" @@ -676,17 +555,10 @@ run_openclaw() { fi } -############################################################################### -# Plugin installation — clone, build, install, enable -# Flow based on openclaw/Dockerfile.e2e -############################################################################### - CLAUDE_MEM_REPO="https://github.com/thedotmack/claude-mem.git" CLAUDE_MEM_BRANCH="${CLI_BRANCH:-main}" PLUGIN_FRESHLY_INSTALLED="" -# Resolve the target extension directory. -# Priority: existing installPath from config > plugins.load.paths > default. resolve_extension_dir() { local oc_config="${HOME}/.openclaw/openclaw.json" if [[ -f "$oc_config" ]] && command -v node &>/dev/null; then @@ -722,12 +594,10 @@ resolve_extension_dir() { CLAUDE_MEM_EXTENSION_DIR="" install_plugin() { - # Check for git before attempting clone check_git CLAUDE_MEM_EXTENSION_DIR="$(resolve_extension_dir)" - # Remove existing plugin installation to allow clean re-install local existing_plugin_dir="$CLAUDE_MEM_EXTENSION_DIR" if [[ -d "$existing_plugin_dir" ]]; then info "Removing existing claude-mem plugin at ${existing_plugin_dir}..." @@ -747,7 +617,6 @@ install_plugin() { local plugin_src="${build_dir}/claude-mem/openclaw" - # Build the TypeScript plugin info "Building TypeScript plugin..." if ! (cd "$plugin_src" && NODE_ENV=development npm install --ignore-scripts 2>&1 && npx tsc 2>&1); then error "Failed to build the claude-mem OpenClaw plugin" @@ -755,7 +624,6 @@ install_plugin() { exit 1 fi - # Create minimal installable package (matches Dockerfile.e2e pattern) local installable_dir="${build_dir}/claude-mem-installable" mkdir -p "${installable_dir}/dist" @@ -763,7 +631,6 @@ install_plugin() { cp "${plugin_src}/dist/index.d.ts" "${installable_dir}/dist/" 2>/dev/null || true cp "${plugin_src}/openclaw.plugin.json" "${installable_dir}/" - # Generate the installable package.json with openclaw.extensions field INSTALLER_PACKAGE_DIR="$installable_dir" node -e " const pkg = { name: 'claude-mem', @@ -775,11 +642,6 @@ install_plugin() { require('fs').writeFileSync(process.env.INSTALLER_PACKAGE_DIR + '/package.json', JSON.stringify(pkg, null, 2)); " - # Clean up stale claude-mem plugin entry before installing. - # If the config references claude-mem but the plugin isn't installed, - # OpenClaw's config validator blocks ALL CLI commands (including plugins install). - # We temporarily remove the entry and save the config so `plugins install` can run, - # then `plugins install` + `plugins enable` will re-create it properly. local oc_config="${HOME}/.openclaw/openclaw.json" local saved_plugin_config="" if [[ -f "$oc_config" ]]; then @@ -808,7 +670,6 @@ install_plugin() { " 2>/dev/null) || true fi - # Install the plugin using OpenClaw's CLI info "Installing claude-mem plugin into OpenClaw..." if ! run_openclaw plugins install "$installable_dir" 2>&1; then error "Failed to install claude-mem plugin" @@ -816,7 +677,6 @@ install_plugin() { exit 1 fi - # Enable the plugin info "Enabling claude-mem plugin..." if ! run_openclaw plugins enable claude-mem 2>&1; then error "Failed to enable claude-mem plugin" @@ -824,9 +684,6 @@ install_plugin() { exit 1 fi - # Ensure claude-mem is present in plugins.allow after successful install+enable. - # Some OpenClaw environments require explicit allowlisting for local plugins. - # This write is guaranteed: if config doesn't exist, configure_memory_slot() will create it. if [[ -f "$oc_config" ]]; then if ! INSTALLER_CONFIG_FILE="$oc_config" node -e " const fs = require('fs'); @@ -845,10 +702,7 @@ install_plugin() { warn "Failed to write plugins.allow — claude-mem may need manual allowlisting" fi else - # Config doesn't exist yet; configure_memory_slot() will create it with plugins.allow - # We'll add claude-mem to the allowlist in a follow-up step after config is materialized info "OpenClaw config not yet materialized; will ensure allowlist in post-install" - # Force config materialization by running a harmless OpenClaw command if run_openclaw status --json >/dev/null 2>&1 && [[ -f "$oc_config" ]]; then if ! INSTALLER_CONFIG_FILE="$oc_config" node -e " const fs = require('fs'); @@ -867,8 +721,6 @@ install_plugin() { fi fi - # Restore saved plugin config (workerPort, syncMemoryFile, observationFeed, etc.) - # from any pre-existing installation that was temporarily removed above. if [[ -n "$saved_plugin_config" && "$saved_plugin_config" != "{}" ]]; then info "Restoring previous plugin configuration..." INSTALLER_CONFIG_FILE="$oc_config" INSTALLER_SAVED_CONFIG="$saved_plugin_config" node -e " @@ -885,23 +737,14 @@ install_plugin() { success "claude-mem plugin installed and enabled" - # ── Copy core plugin files (worker, hooks, scripts) to extension directory ── - # The OpenClaw extension only contains the gateway hook (dist/index.js). - # The actual worker service and Claude Code hooks live in the plugin/ directory - # of the main repo. We copy them so find_claude_mem_install_dir() can locate - # the worker-service.cjs and the worker runs the updated version. local extension_dir="$CLAUDE_MEM_EXTENSION_DIR" local repo_root="${build_dir}/claude-mem" if [[ -d "$extension_dir" && -d "${repo_root}/plugin" ]]; then info "Copying core plugin files to ${extension_dir}..." - # Copy plugin/ directory (worker service, hooks, scripts, skills, UI) cp -R "${repo_root}/plugin" "${extension_dir}/" - # Merge the canonical version from root package.json into the existing - # extension package.json, preserving the openclaw.extensions field that - # plugin discovery requires. local root_version root_version="$(node -e "console.log(require('${repo_root}/package.json').version)")" node -e " @@ -920,11 +763,6 @@ install_plugin() { PLUGIN_FRESHLY_INSTALLED="true" } -############################################################################### -# Memory slot configuration -# Sets plugins.slots.memory = "claude-mem" in ~/.openclaw/openclaw.json -############################################################################### - configure_memory_slot() { local config_dir="${HOME}/.openclaw" local config_file="${config_dir}/openclaw.json" @@ -932,7 +770,6 @@ configure_memory_slot() { mkdir -p "$config_dir" if [[ ! -f "$config_file" ]]; then - # No config file exists — create one with the memory slot info "Creating OpenClaw configuration with claude-mem memory slot..." INSTALLER_CONFIG_FILE="$config_file" node -e " const config = { @@ -955,10 +792,8 @@ configure_memory_slot() { return 0 fi - # Config file exists — update it to set the memory slot info "Updating OpenClaw configuration to use claude-mem memory slot..." - # Use node for reliable JSON manipulation INSTALLER_CONFIG_FILE="$config_file" node -e " const fs = require('fs'); const configPath = process.env.INSTALLER_CONFIG_FILE; @@ -998,11 +833,6 @@ configure_memory_slot() { success "Memory slot set to claude-mem in ${config_file}" } -############################################################################### -# AI Provider setup — interactive provider selection -# Reads defaults from SettingsDefaultsManager.ts (single source of truth) -############################################################################### - AI_PROVIDER="" AI_PROVIDER_API_KEY="" @@ -1026,7 +856,6 @@ setup_ai_provider() { info "AI Provider Configuration" echo "" - # Handle --provider flag (pre-selected via CLI) if [[ -n "$CLI_PROVIDER" ]]; then case "$CLI_PROVIDER" in claude) @@ -1060,7 +889,6 @@ setup_ai_provider() { return 0 fi - # Handle non-interactive mode (no --provider flag) if [[ "$NON_INTERACTIVE" == "true" ]]; then info "Non-interactive mode: defaulting to Claude Max Plan (no API key needed)" AI_PROVIDER="claude" @@ -1124,19 +952,12 @@ setup_ai_provider() { done } -############################################################################### -# Write settings.json — creates ~/.claude-mem/settings.json with all defaults -# Schema: flat key-value (not nested { env: {...} }) -# Defaults sourced from SettingsDefaultsManager.ts -############################################################################### - write_settings() { local settings_dir="${HOME}/.claude-mem" local settings_file="${settings_dir}/settings.json" mkdir -p "$settings_dir" - # Pass provider and API key via environment variables to avoid shell-to-JS injection INSTALLER_AI_PROVIDER="$AI_PROVIDER" \ INSTALLER_AI_API_KEY="$AI_PROVIDER_API_KEY" \ INSTALLER_SETTINGS_FILE="$settings_file" \ @@ -1226,11 +1047,6 @@ write_settings() { success "Settings written to ${settings_file}" } -############################################################################### -# Locate the installed claude-mem plugin directory -# Checks common OpenClaw and Claude Code plugin install paths -############################################################################### - CLAUDE_MEM_INSTALL_DIR="" find_claude_mem_install_dir() { @@ -1250,7 +1066,6 @@ find_claude_mem_install_dir() { fi done - # Fallback: search for the worker script under common plugin roots local -a roots=( "${HOME}/.openclaw" "${HOME}/.claude/plugins" @@ -1260,7 +1075,6 @@ find_claude_mem_install_dir() { local found found="$(find "$root" -name "worker-service.cjs" -path "*/plugin/scripts/*" 2>/dev/null | head -n 1)" || true if [[ -n "$found" ]]; then - # Strip /plugin/scripts/worker-service.cjs to get the install dir CLAUDE_MEM_INSTALL_DIR="${found%/plugin/scripts/worker-service.cjs}" return 0 fi @@ -1271,11 +1085,6 @@ find_claude_mem_install_dir() { return 1 } -############################################################################### -# Worker service startup -# Starts the claude-mem worker using bun in the background -############################################################################### - WORKER_PID="" WORKER_VERSION="" WORKER_AI_PROVIDER="" @@ -1305,7 +1114,6 @@ start_worker() { mkdir -p "$log_dir" - # Ensure bun path is available if [[ -z "$BUN_PATH" ]]; then if ! find_bun_path; then error "Bun not found — cannot start worker service" @@ -1313,12 +1121,10 @@ start_worker() { fi fi - # Start worker in background with nohup CLAUDE_MEM_WORKER_PORT=37777 nohup "$BUN_PATH" "$worker_script" \ >> "$log_file" 2>&1 & WORKER_PID=$! - # Write PID file for future management local pid_file="${HOME}/.claude-mem/worker.pid" mkdir -p "${HOME}/.claude-mem" INSTALLER_PID_FILE="$pid_file" INSTALLER_WORKER_PID="$WORKER_PID" node -e " @@ -1335,13 +1141,6 @@ start_worker() { info "Logs: ${log_file}" } -############################################################################### -# Health verification — two-stage: health (alive) then readiness (initialized) -# Stage 1: Poll /api/health for HTTP 200 (worker process is running) -# Stage 2: Poll /api/readiness for HTTP 200 (worker is fully initialized) -# Total budget: 30 attempts (30 seconds) shared across both stages -############################################################################### - verify_health() { local max_attempts=30 local attempt=1 @@ -1351,7 +1150,6 @@ verify_health() { info "Verifying worker health..." - # ── Stage 1: Wait for /api/health to return HTTP 200 (worker is alive) ── while (( attempt <= max_attempts )); do local http_status http_status="$(curl -s -o /dev/null -w "%{http_code}" "$health_url" 2>/dev/null)" || true @@ -1359,7 +1157,6 @@ verify_health() { if [[ "$http_status" == "200" ]]; then health_alive=true - # Fetch the full health response body and parse metadata local body body="$(curl -s "$health_url" 2>/dev/null)" || true parse_health_json "$body" @@ -1374,7 +1171,6 @@ verify_health() { attempt=$((attempt + 1)) done - # If health never responded, the worker is not running at all if [[ "$health_alive" != "true" ]]; then warn "Worker health check timed out after ${max_attempts} attempts" warn "The worker may still be starting up. Check status with:" @@ -1383,7 +1179,6 @@ verify_health() { return 1 fi - # ── Stage 2: Wait for /api/readiness to return HTTP 200 (fully initialized) ── attempt=$((attempt + 1)) while (( attempt <= max_attempts )); do local readiness_status @@ -1399,17 +1194,12 @@ verify_health() { attempt=$((attempt + 1)) done - # Readiness timed out but health is OK — worker is running, just not fully initialized yet warn "Worker is running but initialization is still in progress" warn "This is normal on first run — the worker will finish initializing in the background." warn "Check readiness with: curl http://127.0.0.1:37777/api/readiness" return 0 } -############################################################################### -# Observation feed setup — optional interactive channel configuration -############################################################################### - FEED_CHANNEL="" FEED_TARGET_ID="" FEED_CONFIGURED=false @@ -1531,10 +1321,6 @@ setup_observation_feed() { FEED_CONFIGURED=true } -############################################################################### -# Write observation feed config into ~/.openclaw/openclaw.json -############################################################################### - write_observation_feed_config() { if [[ "$FEED_CONFIGURED" != "true" ]]; then return 0 @@ -1550,7 +1336,6 @@ write_observation_feed_config() { info "Writing observation feed configuration..." - # Use jq if available, fall back to python3, then node for JSON manipulation if command -v jq &>/dev/null; then local tmp_file tmp_file="$(mktemp)" @@ -1592,7 +1377,6 @@ with open(config_path, 'w') as f: json.dump(config, f, indent=2) " else - # Fallback to node (always available since it's a dependency) INSTALLER_FEED_CHANNEL="$FEED_CHANNEL" \ INSTALLER_FEED_TARGET_ID="$FEED_TARGET_ID" \ INSTALLER_CONFIG_FILE="$config_file" \ @@ -1638,10 +1422,6 @@ with open(config_path, 'w') as f: info "the feed is connected." } -############################################################################### -# Completion summary -############################################################################### - print_completion_summary() { local provider_display="" case "$AI_PROVIDER" in @@ -1661,7 +1441,6 @@ print_completion_summary() { echo -e " ${COLOR_GREEN}✓${COLOR_RESET} Dependencies installed (Bun, uv)" echo -e " ${COLOR_GREEN}✓${COLOR_RESET} OpenClaw gateway detected" - # Show installed version from health data if available if [[ -n "$WORKER_VERSION" ]]; then echo -e " ${COLOR_GREEN}✓${COLOR_RESET} claude-mem v${COLOR_BOLD}${WORKER_VERSION}${COLOR_RESET} installed and running" else @@ -1670,7 +1449,6 @@ print_completion_summary() { echo -e " ${COLOR_GREEN}✓${COLOR_RESET} Memory slot configured" - # Show AI provider with auth method from health data if available if [[ -n "$WORKER_AI_AUTH_METHOD" ]]; then echo -e " ${COLOR_GREEN}✓${COLOR_RESET} AI provider: ${COLOR_BOLD}${WORKER_AI_PROVIDER} (${WORKER_AI_AUTH_METHOD})${COLOR_RESET}" else @@ -1689,7 +1467,6 @@ print_completion_summary() { echo -e " ${COLOR_YELLOW}⚠${COLOR_RESET} Worker may not be running — check logs at ~/.claude-mem/logs/" fi - # Show initialization warning if worker is alive but not yet initialized if [[ "$WORKER_INITIALIZED" != "true" ]] && { [[ -n "$WORKER_REPORTED_PID" ]] || { [[ -n "$WORKER_PID" ]] && kill -0 "$WORKER_PID" 2>/dev/null; }; }; then echo -e " ${COLOR_YELLOW}⚠${COLOR_RESET} Worker is starting but still initializing (this is normal on first run)" fi @@ -1717,16 +1494,11 @@ print_completion_summary() { echo "" } -############################################################################### -# Main -############################################################################### - main() { setup_tty print_banner detect_platform - # --- Step 1: Dependencies --- echo "" info "${COLOR_BOLD}[1/8]${COLOR_RESET} Checking dependencies..." echo "" @@ -1742,12 +1514,10 @@ main() { echo "" success "All dependencies satisfied" - # --- Step 2: OpenClaw gateway --- echo "" info "${COLOR_BOLD}[2/8]${COLOR_RESET} Locating OpenClaw gateway..." check_openclaw - # --- Step 3: Plugin installation (skip if upgrading and already installed) --- echo "" info "${COLOR_BOLD}[3/8]${COLOR_RESET} Installing claude-mem plugin..." @@ -1758,22 +1528,18 @@ main() { install_plugin fi - # --- Step 4: Memory slot configuration --- echo "" info "${COLOR_BOLD}[4/8]${COLOR_RESET} Configuring memory slot..." configure_memory_slot - # --- Step 5: AI provider setup --- echo "" info "${COLOR_BOLD}[5/8]${COLOR_RESET} AI provider setup..." setup_ai_provider - # --- Step 6: Write settings --- echo "" info "${COLOR_BOLD}[6/8]${COLOR_RESET} Writing settings..." write_settings - # --- Step 7: Start worker and verify --- echo "" info "${COLOR_BOLD}[7/8]${COLOR_RESET} Starting worker service..." @@ -1781,8 +1547,6 @@ main() { warn "Port 37777 is already in use (worker may already be running)" info "Checking if the existing service is healthy..." if verify_health; then - # verify_health already called parse_health_json — WORKER_* globals are set. - # Determine the expected version from the installed plugin's package.json. local expected_version="" if [[ -n "$CLAUDE_MEM_INSTALL_DIR" ]] || find_claude_mem_install_dir; then expected_version="$(INSTALLER_PKG="${CLAUDE_MEM_INSTALL_DIR}/package.json" node -e " @@ -1793,8 +1557,6 @@ main() { local needs_restart="" - # If we just installed fresh plugin files, always restart the worker - # to pick up the new version — even if the old worker was healthy. if [[ "$PLUGIN_FRESHLY_INSTALLED" == "true" ]]; then if [[ -n "$WORKER_VERSION" && -n "$expected_version" && "$WORKER_VERSION" != "$expected_version" ]]; then info "Upgrading worker from v${WORKER_VERSION} to v${expected_version}..." @@ -1804,32 +1566,26 @@ main() { needs_restart="true" fi - # Check if worker version is outdated compared to installed version if [[ "$needs_restart" != "true" && -n "$WORKER_VERSION" && -n "$expected_version" && "$WORKER_VERSION" != "$expected_version" ]]; then info "Upgrading worker from v${WORKER_VERSION} to v${expected_version}..." needs_restart="true" fi - # Check if AI provider doesn't match current configuration if [[ "$needs_restart" != "true" && -n "$WORKER_AI_PROVIDER" && -n "$AI_PROVIDER" && "$WORKER_AI_PROVIDER" != "$AI_PROVIDER" ]]; then warn "Worker is using ${WORKER_AI_PROVIDER} but you configured ${AI_PROVIDER} — restarting to apply" needs_restart="true" fi - # Restart worker if needed: kill old process, start fresh if [[ "$needs_restart" == "true" ]]; then info "Stopping existing worker..." - # Try graceful shutdown via API first, fall back to SIGTERM curl -s -X POST "http://127.0.0.1:37777/api/admin/shutdown" >/dev/null 2>&1 || true sleep 2 - # If still running, send SIGTERM to known PID if check_port_37777; then if [[ -n "$WORKER_REPORTED_PID" ]]; then kill "$WORKER_REPORTED_PID" 2>/dev/null || true sleep 1 fi - # Check PID file as fallback local pid_file="${HOME}/.claude-mem/worker.pid" if [[ -f "$pid_file" ]]; then local file_pid @@ -1844,14 +1600,12 @@ main() { fi fi - # Start fresh worker if start_worker; then verify_health || true else warn "Worker restart failed — you can start it manually later" fi else - # No restart needed — show healthy status local uptime_display="" if [[ -n "$WORKER_UPTIME" && "$WORKER_UPTIME" =~ ^[0-9]+$ && "$WORKER_UPTIME" != "0" ]]; then uptime_display="$(format_uptime_ms "$WORKER_UPTIME")" @@ -1888,13 +1642,11 @@ main() { fi fi - # --- Step 8: Observation feed setup (optional) --- echo "" info "${COLOR_BOLD}[8/8]${COLOR_RESET} Observation feed setup..." setup_observation_feed write_observation_feed_config - # --- Completion --- print_completion_summary } diff --git a/openclaw/src/index.test.ts b/openclaw/src/index.test.ts index a2eebe61..9f094aa2 100644 --- a/openclaw/src/index.test.ts +++ b/openclaw/src/index.test.ts @@ -241,7 +241,6 @@ describe("Observation I/O event handlers", () => { body: parsedBody, }); - // Handle different endpoints if (req.url === "/api/health") { res.writeHead(200, { "Content-Type": "application/json" }); res.end(JSON.stringify({ status: "ok" })); @@ -311,7 +310,6 @@ describe("Observation I/O event handlers", () => { sessionId: "test-session-1", }, { sessionKey: "agent-1" }); - // Wait for HTTP request await new Promise((resolve) => setTimeout(resolve, 100)); const initRequest = receivedRequests.find((r) => r.url === "/api/sessions/init"); @@ -358,11 +356,9 @@ describe("Observation I/O event handlers", () => { const { api, fireEvent } = createMockApi({ workerPort }); claudeMemPlugin(api); - // Establish contentSessionId via session_start await fireEvent("session_start", { sessionId: "s1" }, { sessionKey: "test-agent" }); await new Promise((resolve) => setTimeout(resolve, 100)); - // Fire tool result event await fireEvent("tool_result_persist", { toolName: "Read", params: { file_path: "/src/index.ts" }, @@ -420,11 +416,9 @@ describe("Observation I/O event handlers", () => { const { api, fireEvent } = createMockApi({ workerPort }); claudeMemPlugin(api); - // Establish session await fireEvent("session_start", { sessionId: "s1" }, { sessionKey: "summarize-test" }); await new Promise((resolve) => setTimeout(resolve, 100)); - // Fire agent end await fireEvent("agent_end", { messages: [ { role: "user", content: "help me" }, @@ -817,12 +811,10 @@ describe("SSE stream integration", () => { await getService().start({}); - // Wait for connection await new Promise((resolve) => setTimeout(resolve, 200)); assert.ok(logs.some((l) => l.includes("Connecting to SSE stream"))); - // Send an SSE event const observation = { type: "new_observation", observation: { @@ -841,7 +833,6 @@ describe("SSE stream integration", () => { res.write(`data: ${JSON.stringify(observation)}\n\n`); } - // Wait for processing await new Promise((resolve) => setTimeout(resolve, 200)); assert.equal(sentMessages.length, 1); @@ -863,7 +854,6 @@ describe("SSE stream integration", () => { await getService().start({}); await new Promise((resolve) => setTimeout(resolve, 200)); - // Send non-observation events for (const res of serverResponses) { res.write(`data: ${JSON.stringify({ type: "processing_status", isProcessing: true })}\n\n`); res.write(`data: ${JSON.stringify({ type: "session_started", sessionId: "abc" })}\n\n`); @@ -974,8 +964,6 @@ describe("SSE stream integration", () => { }); describe("circuit breaker", () => { - // Reset circuit breaker state before each test by firing gateway_start. - // The circuit is module-level state, so tests would otherwise bleed into each other. beforeEach(async () => { const { api, fireEvent } = createMockApi({ workerPort: 59999 }); claudeMemPlugin(api); @@ -985,17 +973,12 @@ describe("circuit breaker", () => { it("opens after threshold failures and stops further requests", async () => { const { api, logs, fireEvent } = createMockApi({ workerPort: 59999 }); claudeMemPlugin(api); - // Reset circuit inside the test body to guard against timers from preceding - // tests (e.g. completionDelayMs timers) that may fire between beforeEach and here. await fireEvent("gateway_start", {}, {}); - // Fire threshold+1 calls so the circuit is open by the end of the loop - // regardless of whether a concurrent timer fires at the exact boundary. for (let i = 0; i < 4; i++) { await fireEvent("before_agent_start", { prompt: "hello" }, { sessionKey: `cb-open-${i}` }); } - // Circuit is now OPEN. Subsequent calls must be silently dropped. const logCountBeforeDrop = logs.length; await fireEvent("before_agent_start", { prompt: "hello" }, { sessionKey: "cb-drop" }); const noisyDropLogs = logs.slice(logCountBeforeDrop).filter( @@ -1010,18 +993,14 @@ describe("circuit breaker", () => { await fireEvent("gateway_start", {}, {}); const logsAfterReset = logs.length; - // Fire exactly threshold (3) calls for (let i = 0; i < 3; i++) { await fireEvent("before_agent_start", { prompt: "hello" }, { sessionKey: `cb-log-${i}` }); } const newLogs = logs.slice(logsAfterReset); - // At least some failures should have been logged (circuit was active) assert.ok(newLogs.length > 0, "threshold calls should produce log output"); - // Exactly one disabling warning should appear const disablingLogs = newLogs.filter((l) => l.includes("disabling requests")); assert.equal(disablingLogs.length, 1, "should emit exactly one disabling warning when circuit opens"); - // The last call (the threshold-crossing one) should NOT log an individual failure const failureLogs = newLogs.filter((l) => l.includes("failed:")); assert.ok(failureLogs.length < 3, "threshold-crossing call should not log an individual failure"); }); @@ -1031,12 +1010,10 @@ describe("circuit breaker", () => { claudeMemPlugin(api); await fireEvent("gateway_start", {}, {}); - // Open the circuit by firing threshold+1 calls for (let i = 0; i < 4; i++) { await fireEvent("before_agent_start", { prompt: "hello" }, { sessionKey: `cb-reset-${i}` }); } - // Confirm circuit is open (call is silently dropped) const logCountWhileOpen = logs.length; await fireEvent("before_agent_start", { prompt: "hello" }, { sessionKey: "cb-while-open" }); assert.equal( @@ -1045,10 +1022,8 @@ describe("circuit breaker", () => { "call while circuit is open should be silently dropped" ); - // gateway_start resets the circuit await fireEvent("gateway_start", {}, {}); - // Next call should attempt to connect again (not silently drop) const logCountAfterReset = logs.length; await fireEvent("before_agent_start", { prompt: "hello" }, { sessionKey: "cb-after-reset" }); const newLogs = logs.slice(logCountAfterReset); @@ -1059,26 +1034,18 @@ describe("circuit breaker", () => { }); it("HALF_OPEN allows only a single probe — non-2xx keeps circuit open, 2xx closes it", async () => { - // ---- Phase 1: open the circuit via network failures (unreachable port) ---- - // Reset circuit state first const resetMock = createMockApi({ workerPort: 59999 }); claudeMemPlugin(resetMock.api); await resetMock.fireEvent("gateway_start", {}, {}); - // Drive 4 failures to ensure circuit is OPEN for (let i = 0; i < 4; i++) { await resetMock.fireEvent("before_agent_start", { prompt: "probe-test" }, { sessionKey: `probe-phase1-${i}` }); } - // ---- Phase 2: advance clock so cooldown has elapsed ---- - // _circuitOpenedAt was set during Phase 1 using the real Date.now(). - // Advancing Date.now by 31s means the next circuitAllow call sees the cooldown elapsed. const realDateNow = Date.now.bind(Date); Date.now = () => realDateNow() + 31_000; try { - // ---- Phase 3: non-2xx probe — circuit should stay OPEN ---- - // Start a server that returns 500 for all requests let serverA: Server | null = null; const portA: number = await new Promise((resolve) => { serverA = createServer((_req: IncomingMessage, res: ServerResponse) => { @@ -1091,31 +1058,19 @@ describe("circuit breaker", () => { }); }); - // Reuse the same module-level circuit state — just change the worker port. - // Create a new mock api instance pointed at server A (500 responder). const mockA = createMockApi({ workerPort: portA }); claudeMemPlugin(mockA.api); - // Do NOT fire gateway_start here — we want the OPEN circuit state from Phase 1. - // The circuit is OPEN but the mocked clock says cooldown elapsed. - // The next call should: transition to HALF_OPEN, set _halfOpenProbeInFlight=true, - // send the probe to server A (which returns 500), then call circuitOnFailure - // and re-open the circuit. const logCountAtProbe = mockA.logs.length; await mockA.fireEvent("before_agent_start", { prompt: "probe" }, { sessionKey: "probe-call-non2xx" }); await new Promise((resolve) => setTimeout(resolve, 100)); const probeALogs = mockA.logs.slice(logCountAtProbe); - // After a 500 response, circuitOnFailure is called which logs "disabling requests" - // (because state was HALF_OPEN) and logger.warn logs the 500 status. assert.ok( probeALogs.some((l) => l.includes("disabling") || l.includes("returned 500") || l.includes("Worker POST")), "non-2xx probe should keep circuit open (expected disabling or 500 status log)" ); - // Verify probe flag resets: a second call with cooldown elapsed should be allowed as a new probe - // (i.e., _halfOpenProbeInFlight was cleared by circuitOnFailure). - // But without advancing time further the circuit is OPEN again — so calls are dropped. const logCountAfterFailedProbe = mockA.logs.length; await mockA.fireEvent("before_agent_start", { prompt: "probe" }, { sessionKey: "probe-concurrent" }); await new Promise((resolve) => setTimeout(resolve, 100)); @@ -1126,22 +1081,14 @@ describe("circuit breaker", () => { serverA!.close(); - // ---- Phase 4: 2xx probe — circuit should close ---- - // Re-open the circuit with fresh failures, then probe with a 200-returning server. - // Reset circuit state first. const resetMock2 = createMockApi({ workerPort: 59999 }); claudeMemPlugin(resetMock2.api); await resetMock2.fireEvent("gateway_start", {}, {}); - // Drive failures (still using mocked Date.now, but _circuitOpenedAt will be set to - // the mocked time, so cooldown is NOT elapsed yet from the mocked perspective). - // We need to temporarily restore real Date.now while opening the circuit, then - // re-mock it for the probe. Date.now = realDateNow; for (let i = 0; i < 4; i++) { await resetMock2.fireEvent("before_agent_start", { prompt: "probe-test" }, { sessionKey: `probe-phase4-${i}` }); } - // Re-advance the clock past cooldown Date.now = () => realDateNow() + 31_000; let serverB: Server | null = null; @@ -1158,7 +1105,6 @@ describe("circuit breaker", () => { const mockB = createMockApi({ workerPort: portB }); claudeMemPlugin(mockB.api); - // Do NOT fire gateway_start — reuse OPEN circuit state from resetMock2. const logCountBeforeSuccessProbe = mockB.logs.length; await mockB.fireEvent("before_agent_start", { prompt: "probe" }, { sessionKey: "probe-call-2xx" }); diff --git a/openclaw/src/index.ts b/openclaw/src/index.ts index cfe61c3e..5cc0291e 100644 --- a/openclaw/src/index.ts +++ b/openclaw/src/index.ts @@ -1,9 +1,4 @@ -// No file-system imports needed — context is injected via system prompt hook, -// not by writing to MEMORY.md. -// Minimal type declarations for the OpenClaw Plugin SDK. -// These match the real OpenClawPluginApi provided by the gateway at runtime. -// See: https://docs.openclaw.ai/plugin interface PluginLogger { debug?: (message: string) => void; @@ -30,7 +25,6 @@ interface PluginCommandContext { type PluginCommandResult = string | { text: string } | { text: string; format?: string }; -// OpenClaw event types for agent lifecycle interface BeforeAgentStartEvent { prompt?: string; } @@ -136,10 +130,6 @@ interface OpenClawPluginApi { }; } -// ============================================================================ -// SSE Observation Feed Types -// ============================================================================ - interface ObservationSSEPayload { id: number; memory_session_id: string; @@ -166,10 +156,6 @@ interface SSENewObservationEvent { type ConnectionState = "disconnected" | "connected" | "reconnecting"; -// ============================================================================ -// Plugin Configuration -// ============================================================================ - interface FeedEmojiConfig { primary?: string; claudeCode?: string; @@ -193,16 +179,10 @@ interface ClaudeMemPluginConfig { }; } -// ============================================================================ -// Constants -// ============================================================================ - -const MAX_SSE_BUFFER_SIZE = 1024 * 1024; // 1MB +const MAX_SSE_BUFFER_SIZE = 1024 * 1024; const DEFAULT_WORKER_PORT = 37777; const DEFAULT_WORKER_HOST = "127.0.0.1"; -// Emoji pool for deterministic auto-assignment to unknown agents. -// Uses a hash of the agentId to pick a consistent emoji — no persistent state needed. const EMOJI_POOL = [ "🔧","📐","🔍","💻","🧪","🐛","🛡️","☁️","📦","🎯", "🔮","⚡","🌊","🎨","📊","🚀","🔬","🏗️","📝","🎭", @@ -216,7 +196,6 @@ function poolEmojiForAgent(agentId: string): string { return EMOJI_POOL[Math.abs(hash) % EMOJI_POOL.length]; } -// Default emoji values — overridden by user config via observationFeed.emojis const DEFAULT_PRIMARY_EMOJI = "🦞"; const DEFAULT_CLAUDE_CODE_EMOJI = "⌨️"; const DEFAULT_CLAUDE_CODE_LABEL = "Claude Code Session"; @@ -233,19 +212,15 @@ function buildGetSourceLabel( return function getSourceLabel(project: string | null | undefined): string { if (!project) return fallback; - // OpenClaw agent projects are formatted as "openclaw-" if (project.startsWith("openclaw-")) { const agentId = project.slice("openclaw-".length); if (!agentId) return `${primary} openclaw`; const emoji = pinnedAgents[agentId] || poolEmojiForAgent(agentId); return `${emoji} ${agentId}`; } - // OpenClaw project without agent suffix if (project === "openclaw") { return `${primary} openclaw`; } - // Everything else is a Claude Code session. Keep the project identifier - // visible so concurrent sessions can be distinguished in the feed. const trimmedLabel = claudeCodeLabel.trim(); if (!trimmedLabel) { return `${claudeCode} ${project}`; @@ -254,24 +229,12 @@ function buildGetSourceLabel( }; } -// ============================================================================ -// Worker HTTP Client -// ============================================================================ - let _workerHost = DEFAULT_WORKER_HOST; function workerBaseUrl(port: number): string { return `http://${_workerHost}:${port}`; } -// ============================================================================ -// Worker Circuit Breaker -// ============================================================================ -// Prevents CPU-spinning retry loops when the worker is unreachable. -// After CIRCUIT_BREAKER_THRESHOLD consecutive network errors, the circuit -// opens and all worker calls are silently dropped for CIRCUIT_BREAKER_COOLDOWN_MS. -// After the cooldown, one probe attempt is allowed to check if the worker recovered. - const CIRCUIT_BREAKER_THRESHOLD = 3; const CIRCUIT_BREAKER_COOLDOWN_MS = 30_000; @@ -294,7 +257,6 @@ function circuitAllow(logger: PluginLogger): boolean { } return false; } - // HALF_OPEN: allow one probe through if (_halfOpenProbeInFlight) return false; _halfOpenProbeInFlight = true; return true; @@ -429,10 +391,6 @@ async function workerGetJson( } } -// ============================================================================ -// SSE Observation Feed -// ============================================================================ - function formatObservationMessage( observation: ObservationSSEPayload, getSourceLabel: (project: string | null | undefined) => string, @@ -446,8 +404,6 @@ function formatObservationMessage( return message; } -// Explicit mapping from channel name to [runtime namespace key, send function name]. -// These match the PluginRuntime.channel structure in the OpenClaw SDK. const CHANNEL_SEND_MAP: Record = { telegram: { namespace: "telegram", functionName: "sendMessageTelegram" }, whatsapp: { namespace: "whatsapp", functionName: "sendMessageWhatsApp" }, @@ -491,7 +447,6 @@ function sendToChannel( text: string, botToken?: string ): Promise { - // If a dedicated bot token is provided for Telegram, send directly if (botToken && channel === "telegram") { return sendDirectTelegram(botToken, to, text, api.logger); } @@ -514,7 +469,6 @@ function sendToChannel( return Promise.resolve(); } - // WhatsApp requires a third options argument with { verbose: boolean } const args: unknown[] = channel === "whatsapp" ? [to, text, { verbose: false }] : [to, text]; @@ -579,7 +533,6 @@ async function connectToSSEStream( buffer = frames.pop() || ""; for (const frame of frames) { - // SSE spec: concatenate all data: lines with \n const dataLines = frame .split("\n") .filter((line) => line.startsWith("data:")) @@ -620,10 +573,6 @@ async function connectToSSEStream( setConnectionState("disconnected"); } -// ============================================================================ -// Plugin Entry Point -// ============================================================================ - export default function claudeMemPlugin(api: OpenClawPluginApi): void { const userConfig = (api.pluginConfig || {}) as ClaudeMemPluginConfig; const workerPort = userConfig.workerPort || DEFAULT_WORKER_PORT; @@ -638,14 +587,11 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { return baseProjectName; } - // ------------------------------------------------------------------ - // Session tracking for observation I/O - // ------------------------------------------------------------------ const sessionIds = new Map(); const canonicalSessionKeys = new Map(); const sessionAliasesByCanonicalKey = new Map>(); const recentPromptInits = new Map(); - const syncMemoryFile = userConfig.syncMemoryFile !== false; // default true + const syncMemoryFile = userConfig.syncMemoryFile !== false; const syncMemoryFileExclude = new Set(userConfig.syncMemoryFileExclude || []); function getContentSessionId(sessionKey?: string): string { @@ -707,9 +653,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { } const cacheKey = `${contentSessionId}::${project}::${prompt}`; const lastSeenAt = recentPromptInits.get(cacheKey); - // Note: cache is set unconditionally before return. If workerPost fails - // after this check, a retry within 2s would be incorrectly skipped. - // Acceptable because before_agent_start is not retried by the runtime. recentPromptInits.set(cacheKey, now); return typeof lastSeenAt === "number" && now - lastSeenAt <= 2000; } @@ -728,14 +671,10 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { sessionIds.delete(canonicalKey); } - // TTL cache for context injection to avoid re-fetching on every LLM turn. - // before_prompt_build fires on every turn; caching for 60s keeps the worker - // load manageable while still picking up new observations reasonably quickly. const CONTEXT_CACHE_TTL_MS = 60_000; const contextCache = new Map(); async function getContextForPrompt(ctx?: EventContext): Promise { - // Include both the base project and agent-scoped project (e.g. "openclaw" + "openclaw-main") const projects = [baseProjectName]; const agentProject = ctx ? getProjectName(ctx) : null; if (agentProject && agentProject !== baseProjectName) { @@ -743,7 +682,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { } const cacheKey = projects.join(","); - // Return cached context if still fresh const cached = contextCache.get(cacheKey); if (cached && Date.now() - cached.fetchedAt < CONTEXT_CACHE_TTL_MS) { return cached.text; @@ -762,36 +700,21 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { return null; } - // ------------------------------------------------------------------ - // Event: session_start — track session (fires on /new, /reset) - // Init is deferred to before_agent_start to avoid duplicate prompt records. - // ------------------------------------------------------------------ api.on("session_start", async (_event, ctx) => { const { contentSessionId } = rememberSessionContext(ctx); api.logger.info(`[claude-mem] Session tracking initialized: ${contentSessionId}`); }); - // ------------------------------------------------------------------ - // Event: message_received — alias tracking only; init deferred to before_agent_start - // ------------------------------------------------------------------ api.on("message_received", async (event, ctx) => { const { canonicalKey, contentSessionId } = rememberSessionContext(ctx); api.logger.info(`[claude-mem] Message received — prompt capture deferred to before_agent_start: session=${canonicalKey} contentSessionId=${contentSessionId} hasContent=${Boolean(event.content)}`); }); - // ------------------------------------------------------------------ - // Event: after_compaction — preserve session tracking after context compaction. - // Re-init is intentionally NOT called here; the worker retains session state - // independently and re-initializing would create duplicate prompt records. - // ------------------------------------------------------------------ api.on("after_compaction", async (_event, ctx) => { const { contentSessionId } = rememberSessionContext(ctx); api.logger.info(`[claude-mem] Session preserved after compaction: ${contentSessionId}`); }); - // ------------------------------------------------------------------ - // Event: before_agent_start — single init point with dedup guard - // ------------------------------------------------------------------ api.on("before_agent_start", async (event, ctx) => { const { contentSessionId } = rememberSessionContext(ctx); const projectName = getProjectName(ctx); @@ -802,8 +725,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { return; } - // Initialize session in the worker so observations are not skipped - // (the privacy check requires a stored user prompt to exist) await workerPost(workerPort, "/api/sessions/init", { contentSessionId, project: projectName, @@ -813,14 +734,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { api.logger.info(`[claude-mem] Session initialized via before_agent_start: contentSessionId=${contentSessionId} project=${projectName}`); }); - // ------------------------------------------------------------------ - // Event: before_prompt_build — inject context into system prompt - // - // Instead of writing to MEMORY.md (which conflicts with agent-curated - // memory), inject the observation timeline via appendSystemContext. - // This keeps MEMORY.md under the agent's control while still providing - // cross-session context to the LLM. - // ------------------------------------------------------------------ api.on("before_prompt_build", async (_event, ctx) => { if (!shouldInjectContext(ctx)) return; @@ -831,20 +744,15 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { } }); - // ------------------------------------------------------------------ - // Event: tool_result_persist — record tool observations - // ------------------------------------------------------------------ api.on("tool_result_persist", (event, ctx) => { api.logger.info(`[claude-mem] tool_result_persist fired: tool=${event.toolName ?? "unknown"} agent=${ctx.agentId ?? "none"} session=${ctx.sessionKey ?? "none"}`); const toolName = event.toolName; if (!toolName) return; - // Skip memory_ tools to prevent recursive observation loops if (toolName.startsWith("memory_")) return; const { canonicalKey, contentSessionId } = rememberSessionContext(ctx); - // Extract result text from all content blocks let toolResponseText = ""; const content = event.message?.content; if (Array.isArray(content)) { @@ -854,15 +762,11 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { .join("\n"); } - // Truncate long responses to prevent oversized payloads const MAX_TOOL_RESPONSE_LENGTH = 1000; if (toolResponseText.length > MAX_TOOL_RESPONSE_LENGTH) { toolResponseText = toolResponseText.slice(0, MAX_TOOL_RESPONSE_LENGTH); } - // Resolve workspaceDir with fallback chain. - // Empty cwd causes worker-side observation queueing failures, - // so we drop the observation rather than sending cwd: "". const workspaceDir = ctx.workspaceDir; if (!workspaceDir) { @@ -870,7 +774,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { return; } - // Fire-and-forget: send observation to worker workerPostFireAndForget(workerPort, "/api/sessions/observations", { contentSessionId, tool_name: toolName, @@ -880,13 +783,9 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { }, api.logger); }); - // ------------------------------------------------------------------ - // Event: agent_end — summarize session (worker self-completes) - // ------------------------------------------------------------------ api.on("agent_end", async (event, ctx) => { const { contentSessionId } = rememberSessionContext(ctx); - // Extract last assistant message for summarization let lastAssistantMessage = ""; if (Array.isArray(event.messages)) { for (let i = event.messages.length - 1; i >= 0; i--) { @@ -905,25 +804,17 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { } } - // Send summarize. The worker self-completes the session when its SDK-agent - // generator drains; no explicit complete call needed. await workerPost(workerPort, "/api/sessions/summarize", { contentSessionId, last_assistant_message: lastAssistantMessage, }, api.logger); }); - // ------------------------------------------------------------------ - // Event: session_end — clean up session tracking to prevent unbounded growth - // ------------------------------------------------------------------ api.on("session_end", async (_event, ctx) => { clearSessionContext(ctx); api.logger.info(`[claude-mem] Session tracking cleaned up`); }); - // ------------------------------------------------------------------ - // Event: gateway_start — clear session tracking for fresh start - // ------------------------------------------------------------------ api.on("gateway_start", async () => { circuitReset(); sessionIds.clear(); @@ -934,9 +825,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { api.logger.info("[claude-mem] Gateway started — session tracking reset"); }); - // ------------------------------------------------------------------ - // Service: SSE observation feed → messaging channels - // ------------------------------------------------------------------ let sseAbortController: AbortController | null = null; let connectionState: ConnectionState = "disconnected"; let connectionPromise: Promise | null = null; @@ -1014,9 +902,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { return Math.max(1, Math.min(50, Math.trunc(parsed))); } - // ------------------------------------------------------------------ - // Command: /claude_mem_feed — status & toggle - // ------------------------------------------------------------------ api.registerCommand({ name: "claude_mem_feed", description: "Show or toggle Claude-Mem observation feed status", @@ -1050,10 +935,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { }, }); - // ------------------------------------------------------------------ - // Command: /claude-mem-search — query worker search API - // Usage: /claude-mem-search [limit] - // ------------------------------------------------------------------ api.registerCommand({ name: "claude-mem-search", description: "Search Claude-Mem observations by query", @@ -1088,10 +969,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { }, }); - // ------------------------------------------------------------------ - // Command: /claude-mem-recent — recent context snapshot - // Usage: /claude-mem-recent [project] [limit] - // ------------------------------------------------------------------ api.registerCommand({ name: "claude-mem-recent", description: "Show recent Claude-Mem context for a project", @@ -1131,10 +1008,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { }, }); - // ------------------------------------------------------------------ - // Command: /claude-mem-timeline — search and timeline around best match - // Usage: /claude-mem-timeline [depthBefore] [depthAfter] - // ------------------------------------------------------------------ api.registerCommand({ name: "claude-mem-timeline", description: "Find best memory match and show nearby timeline events", @@ -1185,9 +1058,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void { }, }); - // ------------------------------------------------------------------ - // Command: /claude_mem_status — worker health check - // ------------------------------------------------------------------ api.registerCommand({ name: "claude_mem_status", description: "Check Claude-Mem worker health and session status", diff --git a/openclaw/test-e2e.sh b/openclaw/test-e2e.sh index 8af7291d..bec15ce0 100755 --- a/openclaw/test-e2e.sh +++ b/openclaw/test-e2e.sh @@ -1,10 +1,4 @@ #!/usr/bin/env bash -# test-e2e.sh — Run E2E test of claude-mem plugin on real OpenClaw -# -# Usage: -# ./test-e2e.sh # Automated E2E test (build + run + verify) -# ./test-e2e.sh --interactive # Drop into shell for manual testing -# ./test-e2e.sh --build-only # Just build the image, don't run set -euo pipefail cd "$(dirname "$0")" diff --git a/openclaw/test-install.sh b/openclaw/test-install.sh index 888d1224..29e0bec6 100755 --- a/openclaw/test-install.sh +++ b/openclaw/test-install.sh @@ -1,9 +1,6 @@ #!/usr/bin/env bash set -euo pipefail -# Test suite for openclaw/install.sh functions -# Tests the OpenClaw gateway detection, plugin install, and memory slot config. - SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" INSTALL_SCRIPT="${SCRIPT_DIR}/install.sh" @@ -11,10 +8,6 @@ TESTS_RUN=0 TESTS_PASSED=0 TESTS_FAILED=0 -############################################################################### -# Test helpers -############################################################################### - test_pass() { TESTS_RUN=$((TESTS_RUN + 1)) TESTS_PASSED=$((TESTS_PASSED + 1)) @@ -57,30 +50,17 @@ assert_file_exists() { fi } -############################################################################### -# Source the install script without running main() -# We override main to be a no-op, then source the file. -############################################################################### - source_install_functions() { - # Create a temp file that overrides main and sources the install script local tmp_source tmp_source="$(mktemp)" - # Extract everything except the final `main "$@"` invocation sed '$ d' "$INSTALL_SCRIPT" > "$tmp_source" - # Override main to prevent execution echo 'main() { :; }' >> "$tmp_source" - # Source it (suppress color output for cleaner tests) TERM=dumb source "$tmp_source" rm -f "$tmp_source" } source_install_functions -############################################################################### -# Test: detect_platform() — returns a valid platform string -############################################################################### - echo "" echo "=== detect_platform() ===" @@ -118,7 +98,6 @@ test_detect_platform_is_idempotent() { test_detect_platform_is_idempotent test_detect_platform_sets_iswsl_empty_on_non_wsl() { - # Unless actually running on WSL, IS_WSL should be empty PLATFORM="" IS_WSL="" detect_platform >/dev/null 2>&1 @@ -132,15 +111,10 @@ test_detect_platform_sets_iswsl_empty_on_non_wsl() { test_detect_platform_sets_iswsl_empty_on_non_wsl -############################################################################### -# Test: check_bun() — correctly detects bun presence/absence -############################################################################### - echo "" echo "=== check_bun() ===" test_check_bun_detects_installed_bun() { - # If bun is installed on this system, check_bun should succeed if command -v bun &>/dev/null; then BUN_PATH="" if check_bun >/dev/null 2>&1; then @@ -197,15 +171,12 @@ test_find_bun_path_checks_home_bun_bin() { HOME="$fake_home" BUN_PATH="" - # Create a fake bun binary in ~/.bun/bin/ mkdir -p "${fake_home}/.bun/bin" cat > "${fake_home}/.bun/bin/bun" <<'FAKEBUN' -#!/bin/bash echo "1.2.0" FAKEBUN chmod +x "${fake_home}/.bun/bin/bun" - # Hide bun from PATH local saved_path="$PATH" PATH="/nonexistent" @@ -222,15 +193,10 @@ FAKEBUN test_find_bun_path_checks_home_bun_bin -############################################################################### -# Test: check_uv() — correctly detects uv presence/absence -############################################################################### - echo "" echo "=== check_uv() ===" test_check_uv_detects_installed_uv() { - # If uv is installed on this system, check_uv should succeed if command -v uv &>/dev/null; then UV_PATH="" if check_uv >/dev/null 2>&1; then @@ -253,9 +219,6 @@ test_check_uv_detects_installed_uv() { test_check_uv_detects_installed_uv test_check_uv_fails_when_not_found() { - # find_uv_path checks hardcoded system paths (/usr/local/bin/uv, - # /opt/homebrew/bin/uv) that we can't override without root. - # Skip if uv exists at any of those absolute paths. if [[ -x "/usr/local/bin/uv" ]] || [[ -x "/opt/homebrew/bin/uv" ]]; then test_pass "check_uv not-found test: skipped (uv installed at system path)" return 0 @@ -295,15 +258,12 @@ test_find_uv_path_checks_local_bin() { HOME="$fake_home" UV_PATH="" - # Create a fake uv binary in ~/.local/bin/ mkdir -p "${fake_home}/.local/bin" cat > "${fake_home}/.local/bin/uv" <<'FAKEUV' -#!/bin/bash echo "uv 0.4.0" FAKEUV chmod +x "${fake_home}/.local/bin/uv" - # Hide uv from PATH local saved_path="$PATH" PATH="/nonexistent" @@ -320,19 +280,13 @@ FAKEUV test_find_uv_path_checks_local_bin -############################################################################### -# Test: find_openclaw() — not found scenario -############################################################################### - echo "" echo "=== find_openclaw() ===" -# Save original PATH and test with empty locations ORIGINAL_PATH="$PATH" ORIGINAL_HOME="$HOME" test_find_openclaw_not_found() { - # Use a fake HOME where nothing exists local fake_home fake_home="$(mktemp -d)" HOME="$fake_home" @@ -354,7 +308,6 @@ test_find_openclaw_not_found() { test_find_openclaw_not_found -# Test: find_openclaw() — found in HOME/.openclaw/ test_find_openclaw_in_home() { local fake_home fake_home="$(mktemp -d)" @@ -379,10 +332,6 @@ test_find_openclaw_in_home() { test_find_openclaw_in_home -############################################################################### -# Test: configure_memory_slot() — creates new config -############################################################################### - echo "" echo "=== configure_memory_slot() ===" @@ -396,7 +345,6 @@ test_configure_new_config() { local config_file="${fake_home}/.openclaw/openclaw.json" assert_file_exists "$config_file" "Config file created at ~/.openclaw/openclaw.json" - # Verify JSON structure local memory_slot memory_slot="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.slots.memory);")" assert_eq "claude-mem" "$memory_slot" "Memory slot set to claude-mem in new config" @@ -415,13 +363,11 @@ test_configure_new_config() { test_configure_new_config -# Test: configure_memory_slot() — updates existing config test_configure_existing_config() { local fake_home fake_home="$(mktemp -d)" HOME="$fake_home" - # Create an existing config with other settings mkdir -p "${fake_home}/.openclaw" local config_file="${fake_home}/.openclaw/openclaw.json" node -e " @@ -439,22 +385,18 @@ test_configure_existing_config() { configure_memory_slot >/dev/null 2>&1 - # Verify memory slot was updated local memory_slot memory_slot="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.slots.memory);")" assert_eq "claude-mem" "$memory_slot" "Memory slot updated from memory-core to claude-mem" - # Verify existing settings preserved local gateway_mode gateway_mode="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.gateway.mode);")" assert_eq "local" "$gateway_mode" "Existing gateway.mode setting preserved" - # Verify other plugin still present local other_plugin other_plugin="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries['some-other-plugin'].enabled);")" assert_eq "true" "$other_plugin" "Existing plugin entries preserved" - # Verify claude-mem entry was added local cm_enabled cm_enabled="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries['claude-mem'].enabled);")" assert_eq "true" "$cm_enabled" "claude-mem entry added and enabled" @@ -465,7 +407,6 @@ test_configure_existing_config() { test_configure_existing_config -# Test: configure_memory_slot() — preserves existing claude-mem config test_configure_preserves_existing_cm_config() { local fake_home fake_home="$(mktemp -d)" @@ -493,7 +434,6 @@ test_configure_preserves_existing_cm_config() { configure_memory_slot >/dev/null 2>&1 - # Should enable it but preserve existing config local cm_enabled cm_enabled="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries['claude-mem'].enabled);")" assert_eq "true" "$cm_enabled" "claude-mem entry enabled when previously disabled" @@ -512,10 +452,6 @@ test_configure_preserves_existing_cm_config() { test_configure_preserves_existing_cm_config -############################################################################### -# Test: version_gte() — already exists from phase 1 -############################################################################### - echo "" echo "=== version_gte() ===" @@ -537,14 +473,9 @@ else test_fail "version_gte: 1.0.0 < 1.1.14" fi -############################################################################### -# Test: Script structure validation -############################################################################### - echo "" echo "=== Script structure ===" -# Verify all required functions exist for fn in find_openclaw check_openclaw install_plugin configure_memory_slot; do if declare -f "$fn" &>/dev/null; then test_pass "Function ${fn}() is defined" @@ -553,10 +484,8 @@ for fn in find_openclaw check_openclaw install_plugin configure_memory_slot; do fi done -# Verify the CLAUDE_MEM_REPO constant assert_contains "$CLAUDE_MEM_REPO" "github.com/thedotmack/claude-mem" "CLAUDE_MEM_REPO points to correct repository" -# Verify AI provider functions exist for fn in setup_ai_provider write_settings mask_api_key; do if declare -f "$fn" &>/dev/null; then test_pass "Function ${fn}() is defined" @@ -565,10 +494,6 @@ for fn in setup_ai_provider write_settings mask_api_key; do fi done -############################################################################### -# Test: mask_api_key() -############################################################################### - echo "" echo "=== mask_api_key() ===" @@ -581,15 +506,10 @@ assert_eq "****" "$masked_short" "mask_api_key masks keys <= 4 chars entirely" masked_five=$(mask_api_key "12345") assert_eq "*2345" "$masked_five" "mask_api_key masks 5-char key correctly" -############################################################################### -# Test: setup_ai_provider() — non-interactive mode defaults to Claude -############################################################################### - echo "" echo "=== setup_ai_provider() ===" test_setup_ai_provider_non_interactive() { - # NON_INTERACTIVE is readonly, so test in a child bash that sources with --non-interactive local ai_result ai_result="$(bash -c ' set -euo pipefail @@ -609,10 +529,6 @@ test_setup_ai_provider_non_interactive() { test_setup_ai_provider_non_interactive -############################################################################### -# Test: write_settings() — creates new settings.json with defaults -############################################################################### - echo "" echo "=== write_settings() ===" @@ -628,7 +544,6 @@ test_write_settings_new_file() { local settings_file="${fake_home}/.claude-mem/settings.json" assert_file_exists "$settings_file" "settings.json created at ~/.claude-mem/settings.json" - # Verify it's valid JSON with expected defaults local provider provider="$(node -e "const s = JSON.parse(require('fs').readFileSync('${settings_file}','utf8')); console.log(s.CLAUDE_MEM_PROVIDER);")" assert_eq "claude" "$provider" "CLAUDE_MEM_PROVIDER set to claude" @@ -651,7 +566,6 @@ test_write_settings_new_file() { test_write_settings_new_file -# Test: write_settings() — Gemini provider with API key test_write_settings_gemini() { local fake_home fake_home="$(mktemp -d)" @@ -681,7 +595,6 @@ test_write_settings_gemini() { test_write_settings_gemini -# Test: write_settings() — OpenRouter provider with API key test_write_settings_openrouter() { local fake_home fake_home="$(mktemp -d)" @@ -711,13 +624,11 @@ test_write_settings_openrouter() { test_write_settings_openrouter -# Test: write_settings() — preserves existing user customizations test_write_settings_preserves_existing() { local fake_home fake_home="$(mktemp -d)" HOME="$fake_home" - # Create existing settings with custom values mkdir -p "${fake_home}/.claude-mem" local settings_file="${fake_home}/.claude-mem/settings.json" node -e " @@ -730,22 +641,18 @@ test_write_settings_preserves_existing() { require('fs').writeFileSync('${settings_file}', JSON.stringify(settings, null, 2)); " - # Now run write_settings with a new provider AI_PROVIDER="claude" AI_PROVIDER_API_KEY="" write_settings >/dev/null 2>&1 - # Provider should be updated to claude local provider provider="$(node -e "const s = JSON.parse(require('fs').readFileSync('${settings_file}','utf8')); console.log(s.CLAUDE_MEM_PROVIDER);")" assert_eq "claude" "$provider" "Preserve: provider updated to new selection" - # Custom port should be preserved (not overwritten by defaults) local custom_port custom_port="$(node -e "const s = JSON.parse(require('fs').readFileSync('${settings_file}','utf8')); console.log(s.CLAUDE_MEM_WORKER_PORT);")" assert_eq "38888" "$custom_port" "Preserve: existing custom WORKER_PORT preserved" - # Custom log level should be preserved local log_level log_level="$(node -e "const s = JSON.parse(require('fs').readFileSync('${settings_file}','utf8')); console.log(s.CLAUDE_MEM_LOG_LEVEL);")" assert_eq "DEBUG" "$log_level" "Preserve: existing custom LOG_LEVEL preserved" @@ -756,7 +663,6 @@ test_write_settings_preserves_existing() { test_write_settings_preserves_existing -# Test: write_settings() — flat schema has all expected keys test_write_settings_complete_schema() { local fake_home fake_home="$(mktemp -d)" @@ -768,18 +674,15 @@ test_write_settings_complete_schema() { local settings_file="${fake_home}/.claude-mem/settings.json" - # Verify key count matches SettingsDefaultsManager (34 keys) local key_count key_count="$(node -e "const s = JSON.parse(require('fs').readFileSync('${settings_file}','utf8')); console.log(Object.keys(s).length);")" - # Settings should have all 34 keys from SettingsDefaultsManager if (( key_count >= 30 )); then test_pass "Settings file has ${key_count} keys (complete schema)" else test_fail "Settings file has ${key_count} keys, expected >= 30" "Schema may be incomplete" fi - # Verify it does NOT have nested { env: {...} } format local has_env_key has_env_key="$(node -e "const s = JSON.parse(require('fs').readFileSync('${settings_file}','utf8')); console.log(s.env !== undefined);")" assert_eq "false" "$has_env_key" "Settings uses flat schema (no nested 'env' key)" @@ -790,10 +693,6 @@ test_write_settings_complete_schema() { test_write_settings_complete_schema -############################################################################### -# Test: find_claude_mem_install_dir() — not found scenario -############################################################################### - echo "" echo "=== find_claude_mem_install_dir() ===" @@ -817,14 +716,12 @@ test_find_install_dir_not_found() { test_find_install_dir_not_found -# Test: find_claude_mem_install_dir() — found in ~/.openclaw/extensions/claude-mem/ test_find_install_dir_openclaw_extensions() { local fake_home fake_home="$(mktemp -d)" HOME="$fake_home" CLAUDE_MEM_INSTALL_DIR="" - # Create the expected directory structure mkdir -p "${fake_home}/.openclaw/extensions/claude-mem/plugin/scripts" touch "${fake_home}/.openclaw/extensions/claude-mem/plugin/scripts/worker-service.cjs" @@ -841,7 +738,6 @@ test_find_install_dir_openclaw_extensions() { test_find_install_dir_openclaw_extensions -# Test: find_claude_mem_install_dir() — found in ~/.claude/plugins/marketplaces/thedotmack/ test_find_install_dir_marketplace() { local fake_home fake_home="$(mktemp -d)" @@ -864,10 +760,6 @@ test_find_install_dir_marketplace() { test_find_install_dir_marketplace -############################################################################### -# Test: start_worker() — fails gracefully when install dir not found -############################################################################### - echo "" echo "=== start_worker() ===" @@ -892,17 +784,10 @@ test_start_worker_no_install_dir() { test_start_worker_no_install_dir -############################################################################### -# Test: verify_health() — fails when no server is running -############################################################################### - echo "" echo "=== verify_health() ===" test_verify_health_no_server() { - # verify_health should fail gracefully when nothing is running on 37777 - # We use a very short test — just 1 attempt to keep the test fast - # Override the function to test with fewer attempts by running in a subshell local result result="$(bash -c ' set -euo pipefail @@ -912,31 +797,22 @@ test_verify_health_no_server() { echo "main() { :; }" >> "$tmp" source "$tmp" rm -f "$tmp" - # Call verify_health which will attempt 10 polls — capture exit code verify_health 2>/dev/null && echo "PASS" || echo "FAIL" ' 2>/dev/null)" || true - # Note: This test may take ~10 seconds due to polling - # If curl is not available, it will also fail if [[ "$result" == *"FAIL"* ]]; then test_pass "verify_health returns failure when no server is running" else - # Could pass if something is actually running on 37777 test_pass "verify_health returned success (worker may already be running on 37777)" fi } -# Only run the health check test if curl is available if command -v curl &>/dev/null; then test_verify_health_no_server else test_pass "verify_health test skipped (curl not available)" fi -############################################################################### -# Test: print_completion_summary() — runs without error -############################################################################### - echo "" echo "=== print_completion_summary() ===" @@ -987,10 +863,6 @@ test_print_completion_summary_openrouter() { test_print_completion_summary_openrouter -############################################################################### -# Test: Script structure — new functions exist -############################################################################### - echo "" echo "=== New function existence ===" @@ -1002,14 +874,9 @@ for fn in find_claude_mem_install_dir start_worker verify_health print_completio fi done -############################################################################### -# Test: main() function calls new functions in correct order -############################################################################### - echo "" echo "=== main() function structure ===" -# Verify main calls the new functions by checking the install.sh source test_main_calls_start_worker() { if grep -q 'start_worker' "$INSTALL_SCRIPT"; then test_pass "main() calls start_worker" @@ -1070,10 +937,6 @@ test_main_calls_write_observation_feed_config() { test_main_calls_write_observation_feed_config -############################################################################### -# Test: setup_observation_feed() — function exists and non-interactive skips -############################################################################### - echo "" echo "=== setup_observation_feed() ===" @@ -1086,7 +949,6 @@ for fn in setup_observation_feed write_observation_feed_config; do done test_setup_observation_feed_non_interactive() { - # Non-interactive mode should skip feed setup without error local feed_result feed_result="$(bash -c ' set -euo pipefail @@ -1108,10 +970,6 @@ test_setup_observation_feed_non_interactive() { test_setup_observation_feed_non_interactive -############################################################################### -# Test: write_observation_feed_config() — writes correct JSON structure -############################################################################### - echo "" echo "=== write_observation_feed_config() ===" @@ -1120,7 +978,6 @@ test_write_observation_feed_config_writes_json() { fake_home="$(mktemp -d)" HOME="$fake_home" - # Create an existing openclaw.json with claude-mem entry mkdir -p "${fake_home}/.openclaw" local config_file="${fake_home}/.openclaw/openclaw.json" node -e " @@ -1144,7 +1001,6 @@ test_write_observation_feed_config_writes_json() { write_observation_feed_config >/dev/null 2>&1 - # Verify observationFeed was written local feed_enabled feed_enabled="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries['claude-mem'].config.observationFeed.enabled);")" assert_eq "true" "$feed_enabled" "observationFeed.enabled is true" @@ -1157,7 +1013,6 @@ test_write_observation_feed_config_writes_json() { feed_to="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries['claude-mem'].config.observationFeed.to);")" assert_eq "123456789" "$feed_to" "observationFeed.to is 123456789" - # Verify existing config preserved local worker_port worker_port="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries['claude-mem'].config.workerPort);")" assert_eq "37777" "$worker_port" "Existing workerPort preserved after feed config write" @@ -1176,7 +1031,6 @@ test_write_observation_feed_config_skips_when_not_configured() { fake_home="$(mktemp -d)" HOME="$fake_home" - # Create minimal config mkdir -p "${fake_home}/.openclaw" local config_file="${fake_home}/.openclaw/openclaw.json" node -e " @@ -1187,7 +1041,6 @@ test_write_observation_feed_config_skips_when_not_configured() { write_observation_feed_config >/dev/null 2>&1 - # Config should be unchanged — no observationFeed key local has_feed has_feed="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries !== undefined);")" assert_eq "false" "$has_feed" "Config unchanged when FEED_CONFIGURED is false" @@ -1239,14 +1092,9 @@ test_write_observation_feed_config_discord() { test_write_observation_feed_config_discord -############################################################################### -# Test: write_observation_feed_config() — jq/python3/node fallback paths -############################################################################### - echo "" echo "=== write_observation_feed_config() — fallback paths ===" -# Helper: verify feed config JSON was written correctly verify_feed_config_json() { local config_file="$1" expected_channel="$2" expected_target="$3" label="$4" @@ -1262,13 +1110,11 @@ verify_feed_config_json() { feed_to="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries['claude-mem'].config.observationFeed.to);")" assert_eq "$expected_target" "$feed_to" "${label}: observationFeed.to correct" - # Verify existing config preserved local worker_port worker_port="$(node -e "const c = JSON.parse(require('fs').readFileSync('${config_file}','utf8')); console.log(c.plugins.entries['claude-mem'].config.workerPort);")" assert_eq "37777" "$worker_port" "${label}: existing workerPort preserved" } -# Create a seed config file for fallback tests create_seed_config() { local config_file="$1" mkdir -p "$(dirname "$config_file")" @@ -1288,7 +1134,6 @@ create_seed_config() { " } -# Test: jq path (if jq is available) test_write_feed_config_jq_path() { if ! command -v jq &>/dev/null; then test_pass "jq path: skipped (jq not installed)" @@ -1305,7 +1150,6 @@ test_write_feed_config_jq_path() { FEED_TARGET_ID="C01ABC2DEFG" FEED_CONFIGURED="true" - # jq is first in the chain, so just call directly write_observation_feed_config >/dev/null 2>&1 verify_feed_config_json "$config_file" "slack" "C01ABC2DEFG" "jq path" @@ -1319,7 +1163,6 @@ test_write_feed_config_jq_path() { test_write_feed_config_jq_path -# Test: python3 fallback path (hide jq) test_write_feed_config_python3_path() { if ! command -v python3 &>/dev/null; then test_pass "python3 path: skipped (python3 not installed)" @@ -1329,14 +1172,12 @@ test_write_feed_config_python3_path() { local fake_home fake_home="$(mktemp -d)" - # Run in a subshell that hides jq from PATH local result result="$(bash -c ' set -euo pipefail TERM=dumb export HOME="'"$fake_home"'" - # Create seed config using node (node is always available) mkdir -p "'"${fake_home}"'/.openclaw" node -e " const config = { @@ -1353,14 +1194,12 @@ test_write_feed_config_python3_path() { require(\"fs\").writeFileSync(\"'"${fake_home}"'/.openclaw/openclaw.json\", JSON.stringify(config, null, 2)); " - # Source install.sh functions tmp=$(mktemp) sed "$ d" "'"${INSTALL_SCRIPT}"'" > "$tmp" echo "main() { :; }" >> "$tmp" source "$tmp" rm -f "$tmp" - # Hide jq by creating a PATH without it SAFE_PATH="" IFS=":" read -ra path_parts <<< "$PATH" for p in "${path_parts[@]}"; do @@ -1378,7 +1217,6 @@ test_write_feed_config_python3_path() { ' 2>/dev/null)" || true if [[ "$result" == *"DONE"* ]]; then - # Verify the JSON using node local config_file="${fake_home}/.openclaw/openclaw.json" verify_feed_config_json "$config_file" "signal" "+15551234567" "python3 path" else @@ -1390,7 +1228,6 @@ test_write_feed_config_python3_path() { test_write_feed_config_python3_path -# Test: node fallback path (hide both jq and python3) test_write_feed_config_node_path() { local fake_home fake_home="$(mktemp -d)" @@ -1401,7 +1238,6 @@ test_write_feed_config_node_path() { TERM=dumb export HOME="'"$fake_home"'" - # Create seed config mkdir -p "'"${fake_home}"'/.openclaw" node -e " const config = { @@ -1418,22 +1254,12 @@ test_write_feed_config_node_path() { require(\"fs\").writeFileSync(\"'"${fake_home}"'/.openclaw/openclaw.json\", JSON.stringify(config, null, 2)); " - # Create a shadow directory with non-functional jq and python3 - # This makes "command -v" find them but they will fail, so the - # install script will not actually use them successfully. - # However the install script checks "command -v" which just checks - # existence. We need a different approach: override the function - # after sourcing to force the node path. - - # Source install.sh functions tmp=$(mktemp) sed "$ d" "'"${INSTALL_SCRIPT}"'" > "$tmp" echo "main() { :; }" >> "$tmp" source "$tmp" rm -f "$tmp" - # Override write_observation_feed_config to only use the node path - # by extracting just the node branch logic INSTALLER_FEED_CHANNEL="whatsapp" \ INSTALLER_FEED_TARGET_ID="5511999887766@s.whatsapp.net" \ INSTALLER_CONFIG_FILE="'"${fake_home}"'/.openclaw/openclaw.json" \ @@ -1477,7 +1303,6 @@ test_write_feed_config_node_path() { test_write_feed_config_node_path -# Test: write_observation_feed_config uses jq/python3/node fallback chain test_feed_config_fallback_chain_in_source() { if grep -q 'command -v jq' "$INSTALL_SCRIPT"; then test_pass "write_observation_feed_config checks for jq first" @@ -1500,10 +1325,6 @@ test_feed_config_fallback_chain_in_source() { test_feed_config_fallback_chain_in_source -############################################################################### -# Test: print_completion_summary() — shows observation feed status -############################################################################### - echo "" echo "=== print_completion_summary() — observation feed ===" @@ -1547,10 +1368,6 @@ test_completion_summary_without_feed() { test_completion_summary_without_feed -############################################################################### -# Test: Channel type instructions exist in install.sh -############################################################################### - echo "" echo "=== Channel instructions ===" @@ -1562,15 +1379,10 @@ for channel in telegram discord slack signal whatsapp line; do fi done -# Verify specific instruction content assert_contains "$(grep -A2 'userinfobot' "$INSTALL_SCRIPT" 2>/dev/null || echo '')" "userinfobot" "Telegram instructions include @userinfobot" assert_contains "$(grep -A2 'Developer Mode' "$INSTALL_SCRIPT" 2>/dev/null || echo '')" "Developer Mode" "Discord instructions include Developer Mode" assert_contains "$(grep -A2 'C01ABC2DEFG' "$INSTALL_SCRIPT" 2>/dev/null || echo '')" "C01ABC2DEFG" "Slack instructions include sample channel ID" -############################################################################### -# Test: TTY detection — setup_tty() and read_tty() exist -############################################################################### - echo "" echo "=== TTY detection ===" @@ -1582,24 +1394,18 @@ for fn in setup_tty read_tty; do fi done -# Verify TTY_FD is initialized (defaults to 0) if declare -p TTY_FD &>/dev/null; then test_pass "TTY_FD variable is defined" else test_fail "TTY_FD variable should be defined" fi -# Verify setup_tty is called from main() if grep -q 'setup_tty' "$INSTALL_SCRIPT"; then test_pass "main() calls setup_tty" else test_fail "main() should call setup_tty" fi -############################################################################### -# Test: Argument parsing — --provider flag -############################################################################### - echo "" echo "=== Argument parsing — --provider flag ===" @@ -1690,10 +1496,6 @@ test_provider_flag_invalid() { test_provider_flag_invalid -############################################################################### -# Test: Argument parsing — --non-interactive flag (new format) -############################################################################### - echo "" echo "=== Argument parsing — --non-interactive ===" @@ -1740,16 +1542,10 @@ test_non_interactive_with_provider() { test_non_interactive_with_provider -############################################################################### -# Test: --non-interactive mode completes without hanging -############################################################################### - echo "" echo "=== --non-interactive full flow ===" test_non_interactive_completes() { - # Run the full setup_ai_provider + setup_observation_feed in non-interactive mode - # This should complete without any prompts or hangs local result result="$(bash -c ' set -euo pipefail @@ -1772,10 +1568,6 @@ test_non_interactive_completes() { test_non_interactive_completes -############################################################################### -# Test: Script structure — curl | bash usage comment -############################################################################### - echo "" echo "=== curl | bash usage comment ===" @@ -1791,10 +1583,6 @@ else test_fail "install.sh should document --provider flag in usage comment" fi -############################################################################### -# Test: write_settings with --provider flag end-to-end -############################################################################### - echo "" echo "=== write_settings with --provider flag ===" @@ -1836,10 +1624,6 @@ test_write_settings_via_provider_flag() { test_write_settings_via_provider_flag -############################################################################### -# Test: --upgrade flag parsing -############################################################################### - echo "" echo "=== --upgrade flag parsing ===" @@ -1903,10 +1687,6 @@ test_upgrade_not_set_by_default() { test_upgrade_not_set_by_default -############################################################################### -# Test: is_claude_mem_installed() — upgrade detection -############################################################################### - echo "" echo "=== is_claude_mem_installed() ===" @@ -1916,7 +1696,6 @@ test_is_claude_mem_installed_found() { HOME="$fake_home" CLAUDE_MEM_INSTALL_DIR="" - # Create the expected directory structure mkdir -p "${fake_home}/.openclaw/extensions/claude-mem/plugin/scripts" touch "${fake_home}/.openclaw/extensions/claude-mem/plugin/scripts/worker-service.cjs" @@ -1950,15 +1729,10 @@ test_is_claude_mem_installed_not_found() { test_is_claude_mem_installed_not_found -############################################################################### -# Test: check_git() — git availability check -############################################################################### - echo "" echo "=== check_git() ===" test_check_git_available() { - # git should be available in test environment if command -v git &>/dev/null; then local output output="$(check_git 2>&1)" || true @@ -1971,7 +1745,6 @@ test_check_git_available() { test_check_git_available test_check_git_not_available() { - # Test that check_git fails gracefully when git is not in PATH local exit_code=0 PLATFORM="macos" bash -c ' @@ -2035,10 +1808,6 @@ test_check_git_linux_message() { test_check_git_linux_message -############################################################################### -# Test: check_port_37777() — port conflict detection -############################################################################### - echo "" echo "=== check_port_37777() ===" @@ -2052,10 +1821,6 @@ test_check_port_function_exists() { test_check_port_function_exists -############################################################################### -# Test: cleanup_on_exit() — global cleanup trap -############################################################################### - echo "" echo "=== cleanup_on_exit() ===" @@ -2079,7 +1844,6 @@ test_register_cleanup_dir() { local test_dir test_dir="$(mktemp -d)" - # Save existing cleanup dirs local saved_dirs=("${CLEANUP_DIRS[@]+"${CLEANUP_DIRS[@]}"}") CLEANUP_DIRS=() @@ -2091,17 +1855,12 @@ test_register_cleanup_dir() { test_fail "register_cleanup_dir should add directory to CLEANUP_DIRS" fi - # Restore CLEANUP_DIRS=("${saved_dirs[@]+"${saved_dirs[@]}"}") rm -rf "$test_dir" } test_register_cleanup_dir -############################################################################### -# Test: ensure_jq_or_fallback() — JSON utility function -############################################################################### - echo "" echo "=== ensure_jq_or_fallback() ===" @@ -2138,10 +1897,6 @@ test_ensure_jq_with_jq_available() { test_ensure_jq_with_jq_available -############################################################################### -# Test: main() references new functions -############################################################################### - echo "" echo "=== main() references new functions ===" @@ -2205,10 +1960,6 @@ test_usage_comment_includes_upgrade() { test_usage_comment_includes_upgrade -############################################################################### -# Test: Distribution readiness — URL, usage comment, SKILL.md reference -############################################################################### - echo "" echo "=== Distribution readiness ===" @@ -2323,10 +2074,6 @@ test_skill_md_documents_options() { test_skill_md_documents_options -############################################################################### -# Summary -############################################################################### - echo "" echo "========================================" echo "Results: ${TESTS_PASSED}/${TESTS_RUN} passed, ${TESTS_FAILED} failed" diff --git a/openclaw/test-sse-consumer.js b/openclaw/test-sse-consumer.js index eb1e88ec..c73b0431 100644 --- a/openclaw/test-sse-consumer.js +++ b/openclaw/test-sse-consumer.js @@ -1,9 +1,3 @@ -/** - * Smoke test for OpenClaw claude-mem plugin registration. - * Validates the plugin structure works independently of the full OpenClaw runtime. - * - * Run: node test-sse-consumer.js - */ import claudeMemPlugin from "./dist/index.js"; @@ -49,10 +43,8 @@ const mockApi = { }, }; -// Call the default export with mock API claudeMemPlugin(mockApi); -// Verify registration let failures = 0; if (!registeredService) { diff --git a/package.json b/package.json index c565e92f..d8144f01 100644 --- a/package.json +++ b/package.json @@ -61,7 +61,7 @@ "scripts": { "dev": "npm run build-and-sync", "build": "node scripts/sync-plugin-manifests.js && node scripts/build-hooks.js", - "build-and-sync": "npm run build && npm run sync-marketplace && sleep 1 && cd ~/.claude/plugins/marketplaces/thedotmack && npm run worker:restart", + "build-and-sync": "npm run build && npm run sync-marketplace && sleep 1 && (cd ~/.claude/plugins/marketplaces/thedotmack && npm run worker:restart) && npm run queue:clear", "sync-marketplace": "node scripts/sync-marketplace.cjs", "sync-marketplace:force": "node scripts/sync-marketplace.cjs --force", "build:binaries": "node scripts/build-worker-binary.js", @@ -79,6 +79,9 @@ "queue:clear": "bun scripts/clear-failed-queue.ts --all --force", "claude-md:regenerate": "bun scripts/regenerate-claude-md.ts", "claude-md:dry-run": "bun scripts/regenerate-claude-md.ts --dry-run", + "strip-comments": "bun scripts/strip-comments.ts", + "strip-comments:check": "bun scripts/strip-comments.ts --check", + "strip-comments:dry-run": "bun scripts/strip-comments.ts --dry-run", "translate-readme": "bun scripts/translate-readme/cli.ts -v -o docs/i18n README.md", "translate:tier1": "npm run translate-readme -- zh zh-tw ja pt-br ko es de fr", "translate:tier2": "npm run translate-readme -- he ar ru pl cs nl tr uk", @@ -143,7 +146,12 @@ "@types/react": "^19.2.14", "@types/react-dom": "^19.2.3", "esbuild": "^0.28.0", + "jimp": "^1.6.1", "np": "^11.2.0", + "parse5": "^8.0.1", + "postcss": "^8.5.13", + "remark-mdx": "^3.1.1", + "remark-parse": "^11.0.0", "tree-sitter-bash": "^0.25.1", "tree-sitter-c": "^0.24.1", "tree-sitter-cli": "^0.26.8", @@ -165,7 +173,9 @@ "tree-sitter-typescript": "^0.23.2", "ts-prune": "^0.10.3", "tsx": "^4.21.0", - "typescript": "^6.0.3" + "typescript": "^6.0.3", + "unified": "^11.0.5", + "unist-util-visit": "^5.1.0" }, "optionalDependencies": { "tree-kill": "^1.2.2" diff --git a/plans/2026-04-29-installer-streamline.md b/plans/2026-04-29-installer-streamline.md new file mode 100644 index 00000000..120a6a8a --- /dev/null +++ b/plans/2026-04-29-installer-streamline.md @@ -0,0 +1,531 @@ +# Installer Streamline — Eliminate 30s Silent Dead Air + +**Goal:** Move all heavy install work (Bun/uv install, `bun install` in plugin cache) into the `npx claude-mem install` flow with a visible spinner. Make hooks runtime-only — never installers. + +**Net effect:** +- `smart-install.js` runs in normal Claude Code lifecycle: 3 → 0 (or 1 via `npx claude-mem repair` after `claude plugin update`) +- 30s silent dead air → visible spinner during `npx` +- `npx claude-mem repair` becomes the canonical recovery entry point +- ~420 lines of code deleted (smart-install.js × 2 + tests + docs) + +**Out of scope:** `bun-runner.js` deletion (independent rework with Windows/stdin verification needs — ship later). + +--- + +## Phase 0 — Documentation Discovery (already complete) + +These facts came from a discovery agent + direct file reads. Each implementation phase below cites them by line number; do not re-derive. + +### Allowed APIs / patterns to copy + +| Item | Location | What to copy | +|---|---|---| +| NPX command dispatcher | `src/npx-cli/index.ts:39–141` | Manual `switch (command)` on `process.argv.slice(2)`. Each case dynamic-imports its handler. | +| `install` case (template for `repair`) | `src/npx-cli/index.ts:46–52` | `const { runInstallCommand } = await import('./commands/install.js'); await runInstallCommand({ ide: ideValue });` | +| Plugin cache dir helper | `src/npx-cli/utils/paths.ts:32–34` | `pluginCacheDirectory(version)` → `~/.claude/plugins/cache/thedotmack/claude-mem/{version}/` | +| `.install-version` marker readers | `src/services/context/ContextBuilder.ts:36,45` and `src/services/worker/BranchManager.ts:173,228` | These read/delete the marker. Marker schema (`{ version, bun, uv, installedAt }`) MUST be preserved. | +| `clack` task pattern | `src/npx-cli/commands/install.ts:604–664` | `runTasks([{ title, task: async (message) => { … return 'Done OK' } }])` | + +### Anti-patterns / API methods that DO NOT exist (avoid inventing) + +- There is no existing `version-check.js` helper in `plugin/scripts/`. Phase 4 must create it. +- `package.json#files` already globs `plugin/scripts/*.js` (line 50), so deleting `plugin/scripts/smart-install.js` requires no `package.json` change. +- `scripts/smart-install.js` and `plugin/scripts/smart-install.js` are **both source files** kept in sync manually — there is no build step that copies one to the other. Both must be deleted in Phase 5. +- `runSmartInstall()` (install.ts:325–345) shells `node smart-install.js`. After Phase 1 you can call the new module directly — do NOT shell out. +- The `claude plugin install` exec at install.ts:113 has **only one caller** in the entire repo. Safe to remove. + +### File inventory used by this plan + +| File | Lines | Disposition | +|---|---|---| +| `src/npx-cli/commands/install.ts` | 761 | Edited heavily (Phase 2) | +| `src/npx-cli/index.ts` | 147 | One case added (Phase 3) | +| `plugin/hooks/hooks.json` | 93 | Setup hook command rewritten, SessionStart smart-install entry deleted (Phase 4) | +| `scripts/smart-install.js` | 264 | DELETED (Phase 5) | +| `plugin/scripts/smart-install.js` | ≈264 | DELETED (Phase 5) | +| `tests/smart-install.test.ts` | 310 | DELETED (Phase 5) | +| `tests/plugin-scripts-line-endings.test.ts` | 33 | One array entry removed (Phase 5) | +| `plugin/scripts/version-check.js` | NEW | CREATED (Phase 4) | +| `src/npx-cli/install/setup-runtime.ts` | NEW | CREATED (Phase 1) | +| Docs (`docs/public/*.mdx`, `docs/architecture-overview.md`) | misc | Light edit (Phase 6) | + +--- + +## Phase 1 — Create `src/npx-cli/install/setup-runtime.ts` + +**What to implement:** Port the smart-install.js logic to a TypeScript module that takes a target directory parameter (so it can install into the plugin cache dir, not just the marketplace dir). Three exported functions plus internal helpers. + +**File to create:** `src/npx-cli/install/setup-runtime.ts` + +**API surface (these names are used by Phase 2 and Phase 3 — do not rename):** + +```ts +export async function ensureBun(): Promise<{ bunPath: string; version: string }>; +export async function ensureUv(): Promise<{ uvPath: string; version: string }>; +export async function installPluginDependencies(targetDir: string, bunPath: string): Promise; +export function readInstallMarker(targetDir: string): { version: string; bun?: string; uv?: string; installedAt?: string } | null; +export function writeInstallMarker(targetDir: string, version: string, bunVersion: string, uvVersion: string): void; +export function isInstallCurrent(targetDir: string, expectedVersion: string): boolean; +``` + +**Reference implementation to port from:** `scripts/smart-install.js:1–264`. Map old → new: + +| smart-install.js | setup-runtime.ts | +|---|---| +| `getBunPath()` / `isBunInstalled()` / `installBun()` (lines 42–152) | private helpers consumed by `ensureBun()` | +| `getUvPath()` / `isUvInstalled()` / `installUv()` (lines 77–194) | private helpers consumed by `ensureUv()` | +| `needsInstall()` (lines 196–205) | `isInstallCurrent()` + `readInstallMarker()` | +| `installDeps()` (lines 207–226) | `installPluginDependencies(targetDir, bunPath)` — accepts target dir as parameter | +| `verifyCriticalModules()` (lines 228–246) | private helper called inside `installPluginDependencies` | +| `MARKER` constant (line 32) | derive inside each function: `join(targetDir, '.install-version')` | +| Top-level `try { … }` (lines 248–264) | DELETE — caller orchestrates | + +**Key behavioral differences from smart-install.js:** +- All functions take `targetDir` as a parameter (was a top-level `ROOT` constant). +- `ensureBun()` / `ensureUv()` return their version strings rather than logging — caller decides what to display. +- All functions throw on failure with descriptive `Error.message`. The `clack` `runTasks` wrapper in Phase 2 catches and renders. +- `console.error` calls in install/uninstall paths become structured: throw a single `Error` with the manual install instructions in the message body. +- Marker schema is preserved exactly (`{ version, bun, uv, installedAt }`) so existing readers in `ContextBuilder.ts:36` and `BranchManager.ts:173,228` continue to work. + +**Verification checklist:** +- [ ] `bun build src/npx-cli/install/setup-runtime.ts --target=node` succeeds (or whatever the project's TS check command is — confirm via `package.json#scripts`) +- [ ] Marker file format is byte-identical to smart-install.js output (write a marker, diff against a marker written by the old code) +- [ ] `grep -rn "ROOT" src/npx-cli/install/setup-runtime.ts` returns nothing — no top-level constants + +**Anti-pattern guards:** +- ❌ Do not invent a `bunInstall.ts` or `uvInstall.ts` split — keep all three in one file. They share helper code (paths, version probing). +- ❌ Do not import from `plugin/scripts/smart-install.js` — it gets deleted in Phase 5. +- ❌ Do not change the marker schema. Existing readers depend on `{ version }` field. + +--- + +## Phase 2 — Rework `src/npx-cli/commands/install.ts` + +**What to implement:** Drop `needsManualInstall` gating (always run copy/register/enable for every IDE), add a new unconditional "Setting up runtime" task before `setupIDEs`, neuter the claude-code `execSync` shell-out, delete `runSmartInstall()`, and add a `runRepairCommand()` export. + +**File to edit:** `src/npx-cli/commands/install.ts` + +### Edit 2A — Add import for setup-runtime (top of file, after other imports) + +Insert after line 10 (after the `ensureWorkerStarted` import): + +```ts +import { + ensureBun, + ensureUv, + installPluginDependencies, + writeInstallMarker, + isInstallCurrent, +} from '../install/setup-runtime.js'; +``` + +### Edit 2B — Delete `runSmartInstall()` function + +**Delete lines 325–345** (the entire `function runSmartInstall(): boolean { … }` block). + +### Edit 2C — Drop `needsManualInstall` gating, ungate the runTasks block + +**Line 589** currently reads: +```ts +const needsManualInstall = selectedIDEs.some((id) => id !== 'claude-code'); +``` +**Delete line 589.** Update line 593's `if (needsManualInstall) {` to just `{` (or unwrap the block — preferred). The `runTasks` block at lines 604–664 now runs unconditionally. + +**Within that runTasks block:** delete the "Setting up Bun and uv" task entry (lines 656–663). Replace its slot with the new "Setting up runtime" task (Edit 2D). + +### Edit 2D — Insert "Setting up runtime" task + +Replace the deleted "Setting up Bun and uv" task (lines 656–663) with: + +```ts +{ + title: 'Setting up runtime (first install can take ~30s)', + task: async (message) => { + message('Checking Bun…'); + const { version: bunVersion } = await ensureBun(); + message('Checking uv…'); + const { version: uvVersion } = await ensureUv(); + const cacheDir = pluginCacheDirectory(version); + if (!isInstallCurrent(cacheDir, version)) { + message('Installing plugin dependencies…'); + const { bunPath } = await ensureBun(); + await installPluginDependencies(cacheDir, bunPath); + writeInstallMarker(cacheDir, version, bunVersion, uvVersion); + } + return `Runtime ready (Bun ${bunVersion}, uv ${uvVersion}) ${pc.green('OK')}`; + }, +}, +``` + +Place this AFTER the "Installing dependencies" (npm install) task — same ordering position the deleted task occupied. + +### Edit 2E — Neuter the claude-code shell-out in `setupIDEs` + +**Lines 110–123 currently:** +```ts +case 'claude-code': { + try { + execSync( + 'claude plugin marketplace add thedotmack/claude-mem && claude plugin install claude-mem', + { stdio: 'inherit' }, + ); + log.success('Claude Code: plugin installed via CLI.'); + } catch (error: unknown) { + console.error('[install] Claude Code plugin install error:', …); + log.error('Claude Code: plugin install failed. Is `claude` CLI on your PATH?'); + failedIDEs.push(ideId); + } + break; +} +``` + +**Replace with:** +```ts +case 'claude-code': { + log.success('Claude Code: plugin registered (cache + settings written by npx).'); + break; +} +``` + +The cache dir, marketplace registration, plugin registration, and `enabledPlugins` flag have all been written by the (now ungated) runTasks block before `setupIDEs` is called. `claude plugin install` was duplicating that work and triggering the silent Setup hook — both reasons to drop it. + +### Edit 2F — Add `runRepairCommand()` export + +After `runInstallCommand()` (after line 761), append: + +```ts +export async function runRepairCommand(): Promise { + const version = readPluginVersion(); + const cacheDir = pluginCacheDirectory(version); + + if (isInteractive) { + p.intro(pc.bgCyan(pc.black(' claude-mem repair '))); + } else { + console.log('claude-mem repair'); + } + log.info(`Version: ${pc.cyan(version)}`); + + await runTasks([ + { + title: 'Setting up runtime', + task: async (message) => { + message('Checking Bun…'); + const { version: bunVersion } = await ensureBun(); + message('Checking uv…'); + const { version: uvVersion } = await ensureUv(); + message('Reinstalling plugin dependencies…'); + const { bunPath } = await ensureBun(); + await installPluginDependencies(cacheDir, bunPath); + writeInstallMarker(cacheDir, version, bunVersion, uvVersion); + return `Runtime ready (Bun ${bunVersion}, uv ${uvVersion}) ${pc.green('OK')}`; + }, + }, + ]); + + if (isInteractive) { + p.outro(pc.green('claude-mem repair complete.')); + } else { + console.log('claude-mem repair complete.'); + } +} +``` + +`runRepairCommand` always runs the install (no `isInstallCurrent` short-circuit) — the user invoked `repair` because something is wrong, so don't gate on the marker. + +**Verification checklist:** +- [ ] `grep -n "needsManualInstall" src/npx-cli/commands/install.ts` returns nothing +- [ ] `grep -n "runSmartInstall" src/npx-cli/commands/install.ts` returns nothing +- [ ] `grep -n "claude plugin install" src/npx-cli/commands/install.ts` returns nothing +- [ ] `grep -n "claude plugin marketplace add" src/npx-cli/commands/install.ts` returns nothing +- [ ] `runRepairCommand` is exported and TypeScript compiles +- [ ] `runInstallCommand` still exports the same `InstallOptions` shape (Phase 3 needs it untouched) + +**Anti-pattern guards:** +- ❌ Do not delete `runNpmInstallInMarketplace()` — it's still needed for the marketplace dir copy step (other IDEs use that dir). +- ❌ Do not delete `copyPluginToMarketplace()` — non-claude-code IDEs read from `marketplaceDirectory()`. +- ❌ Do not delete the `if (alreadyInstalled)` overwrite-confirm block (lines 538–562) — user-facing UX preserved. + +--- + +## Phase 3 — Wire `npx claude-mem repair` + +**What to implement:** Add a `repair` case to the npx-cli command dispatcher. + +**File to edit:** `src/npx-cli/index.ts` + +### Edit 3A — Add `repair` case + +In the `switch` block (lines 39–141), copy the `install` case pattern from lines 46–52 and adapt: + +```ts +case 'repair': { + const { runRepairCommand } = await import('./commands/install.js'); + await runRepairCommand(); + break; +} +``` + +Place it adjacent to the `install` case for discoverability. + +### Edit 3B — Help text update (if applicable) + +If `src/npx-cli/index.ts` has a help/usage block (look for `case 'help':` or default case), add `repair` to the list of commands with description: `Repair claude-mem runtime (re-runs Bun/uv setup and bun install in plugin cache).` + +**Verification checklist:** +- [ ] `npx claude-mem repair --help` (after build) shows the command +- [ ] `npx claude-mem repair` runs `runRepairCommand` end to end on a corrupted cache (delete `.install-version` then run; should reinstall) +- [ ] Help/usage output (if it exists) lists `repair` + +**Anti-pattern guards:** +- ❌ Do not add CLI flag parsing for `repair` (no flags needed). +- ❌ Do not duplicate the `runRepairCommand` body in `index.ts` — dynamic import only. + +--- + +## Phase 4 — Strip smart-install from hooks; add `version-check.js` + +**What to implement:** Replace the Setup hook's `node smart-install.js` call with a fast version-marker check. Delete the SessionStart smart-install hook entry entirely. + +### Edit 4A — Create `plugin/scripts/version-check.js` + +**File to create:** `plugin/scripts/version-check.js` (new) + +```js +#!/usr/bin/env node +import { existsSync, readFileSync } from 'fs'; +import { join, dirname } from 'path'; +import { fileURLToPath } from 'url'; + +function resolveRoot() { + if (process.env.CLAUDE_PLUGIN_ROOT) { + const root = process.env.CLAUDE_PLUGIN_ROOT; + if (existsSync(join(root, 'package.json'))) return root; + } + try { + const scriptDir = dirname(fileURLToPath(import.meta.url)); + const candidate = dirname(scriptDir); + if (existsSync(join(candidate, 'package.json'))) return candidate; + } catch {} + return null; +} + +const ROOT = resolveRoot(); +if (!ROOT) process.exit(0); + +try { + const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8')); + const markerPath = join(ROOT, '.install-version'); + if (!existsSync(markerPath)) { + console.error('claude-mem: runtime not yet set up — run: npx claude-mem repair'); + process.exit(0); + } + const marker = JSON.parse(readFileSync(markerPath, 'utf-8')); + if (marker.version !== pkg.version) { + console.error(`claude-mem: upgraded to v${pkg.version} — run: npx claude-mem repair`); + } +} catch { + console.error('claude-mem: install marker unreadable — run: npx claude-mem repair'); +} +process.exit(0); +``` + +**Behavior:** +- Sub-100ms (two synchronous file reads + JSON.parse + string compare). +- Always exits 0 (non-blocking) per the project's exit-code strategy in CLAUDE.md. +- Stderr message tells the user exactly what to run if a mismatch is detected. + +### Edit 4B — Rewrite Setup hook command in `plugin/hooks/hooks.json` + +**Lines 4–15** — replace the existing Setup hook command. Current command ends with `node "$_R/scripts/smart-install.js"`. Change it to `node "$_R/scripts/version-check.js"`. Everything before that (PATH export, `_R` resolution, cygpath) stays. + +Concretely: the only change to line 11 is the trailing `smart-install.js` → `version-check.js`. + +### Edit 4C — Delete SessionStart smart-install entry in `plugin/hooks/hooks.json` + +**Lines 17–40** — the SessionStart hook array currently has THREE hook entries: +1. `node "$_R/scripts/smart-install.js"` (lines 21–26) — DELETE this entire entry +2. `node "$_R/scripts/bun-runner.js" "$_R/scripts/worker-service.cjs" start` (lines 27–32) — KEEP +3. `node "$_R/scripts/bun-runner.js" "$_R/scripts/worker-service.cjs" hook claude-code context` (lines 33–38) — KEEP + +After edit, the SessionStart `hooks` array has 2 entries instead of 3. + +**Verification checklist:** +- [ ] `cat plugin/hooks/hooks.json | jq '.hooks.Setup[0].hooks[0].command' | grep version-check.js` succeeds +- [ ] `cat plugin/hooks/hooks.json | jq '.hooks.SessionStart[0].hooks | length'` returns `2` +- [ ] `grep -c "smart-install" plugin/hooks/hooks.json` returns `0` +- [ ] `node plugin/scripts/version-check.js` exits 0 in <500ms (time it) +- [ ] On a fresh checkout (no `.install-version` marker), version-check stderr says "run: npx claude-mem repair" + +**Anti-pattern guards:** +- ❌ Do not change the exit code from 0 — Windows Terminal tab management depends on it (CLAUDE.md exit-code strategy). +- ❌ Do not call out to Bun in version-check.js — Node-only, since this runs before we know Bun exists. +- ❌ Do not add fancy logic (semver compare, partial recovery). String equality is correct: any version mismatch warrants a repair. + +--- + +## Phase 5 — Delete dead code + +**What to implement:** Delete smart-install source files and update tests. + +### Edit 5A — Delete files + +``` +rm scripts/smart-install.js +rm plugin/scripts/smart-install.js +rm tests/smart-install.test.ts +``` + +### Edit 5B — Trim `tests/plugin-scripts-line-endings.test.ts` + +**Line 12 (the `SHEBANG_SCRIPTS` array):** remove the `'smart-install.js'` entry. Keep the rest of the array intact. + +If the array becomes empty after the removal, also remove the entry — but per discovery report it has multiple entries, so just delete the one line. + +### Edit 5C — Add new test for setup-runtime module (optional but recommended) + +**File to create:** `tests/setup-runtime.test.ts` + +Cover: +- `readInstallMarker` returns `null` for missing file +- `writeInstallMarker` produces a JSON object matching the smart-install.js schema (`{ version, bun, uv, installedAt }`) +- `isInstallCurrent` returns `false` for missing marker, `false` for version mismatch, `true` for match +- (Skip Bun/uv install integration tests — those need a sandbox and fall outside this PR's scope.) + +If you skip this, document why in the PR description. + +**Verification checklist:** +- [ ] `find . -name "smart-install*" -not -path "*/node_modules/*"` returns no results +- [ ] `grep -rn "smart-install" tests/` returns no results +- [ ] `npm test` (or whatever the project uses) passes +- [ ] If `tests/setup-runtime.test.ts` was added, it passes + +**Anti-pattern guards:** +- ❌ Do not delete `tests/plugin-scripts-line-endings.test.ts` entirely — it tests other scripts too. +- ❌ Do not delete `tests/bun-runner.test.ts` — bun-runner.js stays in this PR. + +--- + +## Phase 6 — Update docs + +**What to implement:** Sweep documentation to reflect the new install flow. + +### Edit 6A — `docs/architecture-overview.md:36` + +Update reference to smart-install. New copy: "On first install, `npx claude-mem install` sets up Bun and uv globally and runs `bun install` in the plugin cache. The Setup hook then runs a sub-100ms version check on every Claude Code startup; if the plugin was upgraded externally, the user is prompted to run `npx claude-mem repair`." + +### Edit 6B — `docs/public/configuration.mdx:139,163` and `docs/public/development.mdx:42` + +Replace any mention of smart-install behavior with the version-check + repair model. Two-sentence patches; preserve surrounding context. + +### Edit 6C — `docs/public/hooks-architecture.mdx` (11 references) + +This is the largest doc change. Walk each reference (lines 77, 103, 119, 127, 432, 695–696, 703 per discovery report). Update text describing the Setup hook to say it runs `version-check.js` (sub-100ms) instead of `smart-install.js`. Update SessionStart description to reflect 2 entries (worker start + context fetch) instead of 3. + +### Edit 6D — `docs/public/architecture/` references (lines 149, 193) + +Same pattern — replace smart-install lifecycle description with the npx-installer + version-check model. + +### Edit 6E — Skip CHANGELOG + +CLAUDE.md says: "No need to edit the changelog ever, it's generated automatically." Don't touch it. + +### Edit 6F — Skip docs/reports/ + +Those are historical incident reports. Do not retroactively edit them — they describe past behavior. + +**Verification checklist:** +- [ ] `grep -rn "smart-install" docs/public/` returns no results +- [ ] `grep -rn "smart-install" docs/architecture-overview.md` returns no results +- [ ] (Optional) Render docs locally via Mintlify dev server and visually scan the architecture page + +**Anti-pattern guards:** +- ❌ Do not edit `docs/reports/*.md` — those are dated incident reports, leave them alone. +- ❌ Do not edit CHANGELOG.md. + +--- + +## Phase 7 — Build, test, manual verify + +**What to implement:** End-to-end validation. This phase is run by the implementer before opening the PR. + +### Edit 7A — Build + +```bash +npm run build-and-sync +``` + +This must succeed. If TypeScript fails on the new `setup-runtime.ts`, fix in place. + +### Edit 7B — Test suite + +```bash +npm test +``` + +Must be green. Likely failures to anticipate: +- `plugin-scripts-line-endings.test.ts` if the `'smart-install.js'` entry was missed in Phase 5 +- Any test that imports from `scripts/smart-install.js` (discovery report says only `tests/smart-install.test.ts`, which Phase 5 deletes) + +### Edit 7C — Manual fresh-install verification + +1. On a clean machine (or after `rm -rf ~/.claude/plugins/marketplaces/thedotmack ~/.claude/plugins/cache/thedotmack ~/.claude-mem`): + ```bash + npx claude-mem install + ``` + Confirm: + - Spinner says "Setting up runtime (first install can take ~30s)" + - No silent dead air + - Worker starts at the end +2. Open Claude Code in any project. Confirm: + - Setup hook fires fast (<200ms total) + - SessionStart fires fast (no smart-install delay) + - No "claude plugin install" output +3. Simulate a stale install: + ```bash + rm ~/.claude/plugins/cache/thedotmack/claude-mem//.install-version + ``` + Open a new Claude Code session. Confirm version-check.js prints the "run: npx claude-mem repair" message to stderr. +4. Run repair: + ```bash + npx claude-mem repair + ``` + Confirm spinner runs through Bun/uv check + bun install + marker write, then exits clean. + +### Edit 7D — Commit and open PR + +Per the PR creation flow in the user's outer task. Don't auto-merge; the user wants a review loop. + +**Verification checklist:** +- [ ] `npm run build-and-sync` exits 0 +- [ ] `npm test` exits 0 +- [ ] Manual fresh install completes with visible spinner, no silent dead air +- [ ] Setup hook fires <200ms after rebuild +- [ ] `npx claude-mem repair` runs end-to-end + +**Anti-pattern guards:** +- ❌ Do not skip the manual verification — the whole point of this PR is UX (eliminating dead air). Type checks won't catch a regression. +- ❌ Do not bump the version — version bump is handled separately by the version-bump skill. + +--- + +## Summary of file changes + +| Type | Path | Phase | +|---|---|---| +| Created | `src/npx-cli/install/setup-runtime.ts` | 1 | +| Edited | `src/npx-cli/commands/install.ts` | 2 | +| Edited | `src/npx-cli/index.ts` | 3 | +| Created | `plugin/scripts/version-check.js` | 4 | +| Edited | `plugin/hooks/hooks.json` | 4 | +| Deleted | `scripts/smart-install.js` | 5 | +| Deleted | `plugin/scripts/smart-install.js` | 5 | +| Deleted | `tests/smart-install.test.ts` | 5 | +| Edited | `tests/plugin-scripts-line-endings.test.ts` | 5 | +| Created | `tests/setup-runtime.test.ts` (optional) | 5 | +| Edited | `docs/architecture-overview.md` | 6 | +| Edited | `docs/public/configuration.mdx` | 6 | +| Edited | `docs/public/development.mdx` | 6 | +| Edited | `docs/public/hooks-architecture.mdx` | 6 | +| Edited | `docs/public/architecture/*.md` | 6 | + +Estimated diff: **+250 / −500 lines** (net deletion). diff --git a/plans/2026-04-30-onboarding-ux-overhaul.md b/plans/2026-04-30-onboarding-ux-overhaul.md new file mode 100644 index 00000000..0338fc14 --- /dev/null +++ b/plans/2026-04-30-onboarding-ux-overhaul.md @@ -0,0 +1,367 @@ +# Onboarding UX Overhaul + +Three surfaces, one product voice, one first-success moment. Each phase is self-contained and can be executed in a fresh chat with `/do`. + +## North Star + +Pull the user toward this single moment: **open the viewer in a browser, do anything in Claude Code, watch an observation appear within seconds.** All three surfaces aim at it from different angles. + +## Cross-Cutting Facts (read this first, every phase) + +- **Test runner:** `bun test`. Test command: `npm run test`. Tests live in `tests/`. Pattern templates: `tests/sqlite/observations.test.ts:1-60` (in-memory SQLite + bun:test), `tests/install-non-tty.test.ts:1-95` (regex assertions over install.ts source). +- **Build:** `npm run build-and-sync` runs full build (banner frames + plugin manifests + `scripts/build-hooks.js`) → marketplace sync → worker restart. Viewer compiles via esbuild to `plugin/ui/viewer-bundle.js`; HTML template (which holds ALL CSS) at `src/ui/viewer-template.html`. +- **Settings defaults:** `src/shared/SettingsDefaultsManager.ts:70-131`. Merge logic at `loadFromFile()` lines 161-205 — missing keys auto-pick up new defaults, explicit values are respected. Forward-compatible. +- **`CLAUDE_MEM_WELCOME_HINT_ENABLED` already defaults to `'true'`** (`SettingsDefaultsManager.ts:104`). Single reader at `SearchRoutes.ts:294`. Goal 5 from the brief is already done — we replace "flip the default" with "pin it with a regression test." +- **Timing line, identical wording everywhere:** `Memory injection starts on your second session in a project.` +- **Privacy line, identical wording everywhere:** `Everything stays in ~/.claude-mem on this machine.` + +--- + +## Phase 0 — Documentation Discovery (DONE; for reference) + +Discovery already completed. Allowed APIs and signatures established: + +### Install.ts patterns (`src/npx-cli/commands/install.ts`) +- `log` helper at lines 41-46 — methods `info | success | warn | error`, conditionally routes to `p.log.*` (interactive) vs `console.log/warn/error` (non-interactive, 2-space indent). +- `p` is `* as p from '@clack/prompts'`. Used: `p.note(body, title)`, `p.outro(msg)`, `p.intro`, `p.log.*`, `p.tasks`, `p.spinner`, `p.select/multiselect/confirm/password`, `p.isCancel`, `p.cancel`. +- `pc` is `picocolors` default import. Available: `pc.cyan/green/yellow/red/bold/underline/dim/bgCyan/black`. **`pc.dim` exists** (already in use at line 663). +- `getSetting('CLAUDE_MEM_WORKER_PORT')` returns string; convert with `Number()` when needed. +- Health probe pattern at lines 843-864: `fetch('http://127.0.0.1:${port}/api/health', { signal: AbortSignal.timeout(3000) })`, non-throwing. +- Existing `summaryLines` block (826-841) and `nextSteps` block (866-896) — both have parallel interactive (`p.note`) and non-interactive (`console.log`) branches. + +### Settings (`src/shared/SettingsDefaultsManager.ts`) +- `CLAUDE_MEM_WELCOME_HINT_ENABLED: 'true'` at line 104. +- Merge: defaults first, then file overrides, then env overrides (lines 194-201). +- Install does NOT pre-seed this key — only seeds prompted settings (provider, model). Existing users without explicit value automatically get the new default. +- `SettingsRoutes.ts:84-117` — flag is NOT in the user-updatable allowlist (read-only via UI). +- Test template: `tests/install-non-tty.test.ts` (regex over source); SettingsDefaultsManager has no dedicated test file — would be created if needed. + +### SessionStart hint (`src/services/worker/http/routes/SearchRoutes.ts`) +- `WELCOME_HINT_TEMPLATE` at lines 14-27. Used at line 301: `WELCOME_HINT_TEMPLATE.replace('{viewer_url}', viewerUrl)`. +- Gating logic at lines 293-306. Only fires when `hintEnabled && !full && observationCount === 0`. +- Output is plain text injected as SessionStart `additionalContext` via the SessionStart hook (`src/cli/handlers/context.ts`). + +### Viewer (`src/ui/viewer/`) +- `useSSE()` at `src/ui/viewer/hooks/useSSE.ts:1-148` exposes `{ observations, summaries, prompts, projects, sources, projectsBySource, isProcessing, queueDepth, isConnected }`. Auto-reconnects; new observations prepended via `'new_observation'` SSE event. +- `WelcomeCard` mounted in `src/ui/viewer/App.tsx:128-130`, currently receives only `onDismiss`. App has access to all SSE state (lines 51-67). +- All viewer CSS lives in `src/ui/viewer-template.html`; existing `.welcome-card*` styles at lines 1443-1561; existing `.status-dot` + `@keyframes pulse` at lines 754-764. +- Stats endpoints: `/api/stats` (`DataRoutes.ts:204-242`) returns `{database: {observations, sessions, summaries}, worker: {...}}`. `/api/projects` (`DataRoutes.ts:244-260`) returns `ProjectCatalog`. **No `firstObservationAt` field currently — Phase 4 adds it.** +- `/api/how-it-works` is NOT a static explainer — it queries observations tagged with the `'how-it-works'` concept (`SearchManager.ts:836-884`). Useless on a fresh install. Phase 1 adds a true static explainer. + +### Skills (`plugin/skills/`) +- 9 existing skills, each a directory with `SKILL.md` and YAML frontmatter (`name`, `description`). No central registry — discovered by directory convention. +- Template to copy: `plugin/skills/mem-search/SKILL.md`. + +### Anti-patterns to avoid +- DO NOT call `/api/how-it-works` for an onboarding explainer — wrong endpoint. +- DO NOT add new viewer CSS files — all styles in `src/ui/viewer-template.html`. +- DO NOT add new viewer routes for stats unless strictly needed — extend `/api/stats` instead. +- DO NOT seed `CLAUDE_MEM_WELCOME_HINT_ENABLED` in `install.ts` — defaults already handle it. +- DO NOT pass imperatives ("you should run X") in the SessionStart hint — Claude will try to execute. Use third-person narration ("`/learn-codebase` is available if…"). + +--- + +## Phase 1 — Canonical Onboarding Explainer + +**Why:** All three surfaces need a single source of truth for the 90-second "what is this" explainer. `/api/how-it-works` does not serve this purpose. We'll create a real static explainer and link to it from everywhere. + +### Tasks + +1. Create `src/services/worker/onboarding-explainer.md` — single canonical content. ~150 words, three sections: + - **What it does:** Every Read/Edit/Bash Claude makes turns into a compressed observation. Observations get summarized at session end. Relevant ones get auto-injected into future prompts. + - **When it kicks in:** Memory injection starts on your second session in a project. *(verbatim timing line)* + - **Where data lives:** Everything stays in ~/.claude-mem on this machine. *(verbatim privacy line)* + +2. Add new route `GET /api/onboarding/explainer` in `src/services/worker/http/routes/SearchRoutes.ts`: + - Read the markdown file at boot (cache like `cachedSkillMd` pattern in `Server.ts:18-33`). + - Serve as `text/markdown; charset=utf-8`. + - Register in `setupRoutes()` next to the other `/api/context/*` routes. + +3. Create `plugin/skills/how-it-works/SKILL.md`: + - Copy frontmatter shape from `plugin/skills/mem-search/SKILL.md:1-4`. + - `name: how-it-works` + - `description: Explain how claude-mem captures observations, when memory injection kicks in, and where data lives. Use when the user asks "how does claude-mem work?" or "what is this thing doing?".` + - Body: same content as the markdown explainer (or fetch `/api/onboarding/explainer` at runtime). + - Wire into `scripts/build-hooks.js` verification list (lines 336-348) so build fails if the file is missing. + +### Verification + +- `npm run build-and-sync` succeeds; new SKILL.md present in `plugin/skills/how-it-works/`. +- `curl http://localhost:$PORT/api/onboarding/explainer` returns the markdown. +- Worker boot log includes a "Cached onboarding explainer at boot" entry (mirroring the SKILL.md cache log). + +### Anti-pattern guards + +- Do NOT alter `/api/how-it-works`. It serves a different (concept-tagged search) purpose. +- Do NOT inline the explainer text into install.ts / WelcomeCard / WELCOME_HINT_TEMPLATE — link, don't duplicate. + +--- + +## Phase 2 — SessionStart Welcome Hint Rewrite + +**Why:** Current copy reads as a marketing intercept inside Claude's context, leads with imperatives Claude tries to execute, and doesn't set the truthful "today seeds, tomorrow injects" expectation. + +### Tasks + +1. Rewrite `WELCOME_HINT_TEMPLATE` at `src/services/worker/http/routes/SearchRoutes.ts:14-27`. Target: + + ``` + # claude-mem status + + This project has no memory yet. The current session will seed it; subsequent sessions will receive auto-injected context for relevant past work. + + Memory injection starts on your second session in a project. + + `/learn-codebase` is available if the user wants to front-load the entire repo into memory in a single pass (~5 minutes on a typical repo, optional). Otherwise memory builds passively as work happens. + + Live activity: {viewer_url} + How it works: `/how-it-works` + + This message disappears once the first observation lands. + ``` + + Constraints: third-person narration referring to "the user", not imperatives directed at Claude. Title is "status", not "Welcome". + +2. **Pin the default with a test.** In a new file `tests/shared/welcome-hint-default.test.ts`: + - Assert `SettingsDefaultsManager.getAllDefaults().CLAUDE_MEM_WELCOME_HINT_ENABLED === 'true'`. + - Assert that an empty settings file resolves to `'true'`. + - Assert that an explicit `'false'` is preserved through `loadFromFile`. + +3. No install.ts seeding change — defaults already flow through. + +4. Audit existing welcome-hint tests (memory note: "4/4 tests pass"). Likely in `tests/worker/SearchManager.timeline-anchor.test.ts` per discovery; if those tests assert the old template body verbatim, update them to match the new copy. If they only assert the gating logic, leave alone. + +### Verification + +- `bun test tests/shared/welcome-hint-default.test.ts` passes. +- `bun test tests/worker/` (or whichever file holds the welcome-hint tests) passes. +- Manual: in a fresh project with zero observations, start a Claude Code session — SessionStart context includes the new status note. New text contains the verbatim timing line and points at `{viewer_url}` and `/how-it-works`. +- Manual: in a project with observations, the hint does NOT appear (gating still works). + +### Anti-pattern guards + +- Do NOT use the word "Welcome" or any second-person imperatives ("you should…", "go to…"). Claude will try to "help" by executing them. +- Do NOT exceed ~10 lines — this is injected into Claude's context for every fresh-project session. + +--- + +## Phase 3 — Post-Install Next Steps Rewrite + +**Why:** Current 4-bullet menu treats `/learn-codebase`, `/mem-search`, and `/knowledge-agent` as parallel options. They aren't — `/learn-codebase` is the only first-session move and even it's optional. Lead with proof (live viewer), give two paths, defuse the privacy concern. + +### Tasks + +1. Replace the `nextSteps` array at `src/npx-cli/commands/install.ts:866-878`. Target body when worker is ready: + + ``` + ${pc.green('✓')} Worker running at ${pc.underline(`http://localhost:${actualPort}`)} + + ${pc.bold('First success:')} keep that URL open in a browser, then open Claude Code in any project. Observations stream in as Claude reads, edits, and runs commands. + + ${pc.bold('Two paths from here:')} + ${pc.cyan('A.')} Just start working. Memory builds passively from your first prompt. (Recommended.) + ${pc.cyan('B.')} Front-load it: open Claude Code and run ${pc.bold('/learn-codebase')} to ingest the whole repo (~5 min, optional). + + Memory injection starts on your second session in a project. + Everything stays in ${pc.cyan('~/.claude-mem')} on this machine. + + ${pc.dim('How it works: /how-it-works · Disable first-session hint: CLAUDE_MEM_WELCOME_HINT_ENABLED=false')} + ${pc.dim('Note: close all Claude Code sessions before uninstalling, or ~/.claude-mem will be recreated by active hooks.')} + ``` + + Worker-not-ready branch: keep the existing `pc.yellow('!')` warning + retry hint, then append the same "First success" / "Two paths" / timing / privacy lines (substituting `workerPort` for `actualPort`). + +2. Drop `/mem-search` and `/knowledge-agent` lines from this surface entirely. (They reappear in WelcomeCard for users who do open the viewer.) + +3. Keep both `isInteractive` (uses `p.note(nextSteps.join('\n'), 'Next Steps')`) and non-interactive (`console.log` per line, 2-space indent) branches in sync. The array shape stays the same — only the strings change. + +4. Verify `pc.dim` renders correctly under the clack `p.note` box (it does — line 663 already uses it). + +### Verification + +- `npm run build` succeeds. +- Manual interactive run: `npx claude-mem install` in a fresh dir shows the new Next Steps block inside the clack box. +- Manual non-interactive run: `CI=true npx claude-mem install` (or pipe through cat) shows the same content with 2-space indent and no clack boxes. +- Update `tests/install-non-tty.test.ts` regex assertions to match the new strings (existing pattern: `expect(installSource).toContain(...)`). + +### Anti-pattern guards + +- Do NOT add new commands to this surface. The point is reduction. +- Do NOT lose the uninstall caveat — demote, don't delete. +- Do NOT reorder so the worker URL becomes a footnote — it's the single most important payload here. + +--- + +## Phase 4 — Extend `/api/stats` with `firstObservationAt` + +**Why:** The viewer micro-stat row (Phase 5) needs a "since [date]" value. No HTTP endpoint currently exposes the earliest observation timestamp. Smallest possible backend change to enable Phase 5. + +### Tasks + +1. Add a `firstObservationAt: string | null` field to the stats response in `src/services/worker/http/routes/DataRoutes.ts:204-242` (`handleGetStats`). + +2. Add a SQL helper next to `getRecentObservations` (`src/services/sqlite/observations/recent.ts:6-20`): + + ```ts + export function getFirstObservationCreatedAt(db: SessionStore): string | null { + // SELECT created_at FROM observations ORDER BY created_at_epoch ASC LIMIT 1 + } + ``` + + Match the existing prepared-statement pattern in that directory. + +3. Wire the helper into `handleGetStats` and surface as ISO string (or `null` if no observations). Verify the existing TypeScript type for the stats response is updated. + +### Verification + +- `bun test tests/sqlite/observations.test.ts` still passes. +- New unit test in `tests/sqlite/observations.test.ts` (or a new file) covering `getFirstObservationCreatedAt` for empty + non-empty DB. +- `curl http://localhost:$PORT/api/stats` returns the new field. + +### Anti-pattern guards + +- Do NOT add new endpoints — extend the existing `/api/stats` payload. +- Do NOT add per-project earliest-timestamp logic; the viewer stat row is global ("X observations · Y projects · since [date]"). + +--- + +## Phase 5 — Viewer WelcomeCard Rewrite + +**Why:** Current card is generic and doesn't differentiate the empty state (the moment the user is asking "is anything happening?") from the data state (the moment the user is asking "what can I do here?"). + +### Tasks + +1. **App.tsx wiring** (`src/ui/viewer/App.tsx:128-130`). Pass new props to `WelcomeCard`: + ```tsx + + ``` + If a stats fetch hook doesn't already exist, add one (`useStats()` at `src/ui/viewer/hooks/useStats.ts`) that polls `/api/stats` on mount and on each new SSE observation. + +2. **WelcomeCard.tsx rewrite** (`src/ui/viewer/components/WelcomeCard.tsx`): + - Bump localStorage key to `claude-mem-welcome-dismissed-v2` (keep helpers in same file). v1 dismissals should NOT carry over — the card is meaningfully different. + - Branch on `observationCount === 0`: + - **Empty state:** + - Headline: "No observations yet." + - Body: "Open Claude Code in any project — entries stream in here as Claude reads, edits, and runs commands." + - Live status row with a `` and label "Connected to worker · waiting for activity" / "Reconnecting…" based on `isConnected`. + - Footer: "How it works" link + dismiss button (existing behavior). + - **Has-data state:** + - Headline: "claude-mem" + - Body: "Persistent memory across Claude Code sessions." + - Stat row: `${observationCount} observations · ${projectCount} projects · since ${formatDate(firstObservationAt)}`. + - Two example prompts (cut from four): + - `ask: did we already solve X?` + - `/mem-search dig into past work` + - Footer: "How it works" + "Read the docs" links + dismiss. + - "How it works" link points to `/api/onboarding/explainer` (opens in new tab as raw markdown — acceptable for v1; or a small modal showing the markdown rendered). + +3. **CSS additions** in `src/ui/viewer-template.html` next to `.welcome-card-*` styles (lines 1443-1561). Reuse existing `@keyframes pulse` (line 754). Add: + - `.welcome-card-status-dot` (8×8 circle, error color + pulse when disconnected, success color + no animation when `data-connected="true"`). + - `.welcome-card-stats` (single-row, dim text, dot separators using `·`). + - `.welcome-card-empty` adjustments (slightly larger lede, status row layout). + +4. **Auto-dismiss on first observation:** in App.tsx, add an effect that flips the card from empty→has-data view automatically when `observationCount` crosses 0→1. The card should NOT auto-dismiss permanently on the first observation — it just transitions states. The user explicitly dismisses with the X. + +### Verification + +- `npm run build-and-sync` succeeds; viewer bundle rebuilds. +- Open viewer in a fresh-install state: empty card shows, dot animates (or is solid green if connected). +- In Claude Code, do one Read in a project. The viewer card flips to has-data state without a manual refresh, stat row populates. +- Dismiss persists across reload (localStorage v2 key). +- Header "Show help" button still re-opens the card. +- Tests: a small unit test for `getStoredWelcomeDismissed` / `setStoredWelcomeDismissed` against the new v2 key (extend the helper logic — pure functions are easy to test even without React Testing Library). + +### Anti-pattern guards + +- Do NOT add a new CSS file. All styles in `viewer-template.html`. +- Do NOT poll `/api/stats` on every render — once on mount + on `'new_observation'` SSE event is enough. +- Do NOT auto-permanently-dismiss on first observation; users may want to keep the card visible for the example prompts. +- Do NOT inline the explainer text into the card — link to `/api/onboarding/explainer`. + +--- + +## Phase 6 — Drift Audit + +**Why:** Three surfaces, two verbatim lines, one explainer source. Catch any divergence before it ships. + +### Tasks + +1. Grep for the timing line and assert it appears verbatim in: + - `src/services/worker/http/routes/SearchRoutes.ts` (Phase 2) + - `src/npx-cli/commands/install.ts` (Phase 3) + - `src/services/worker/onboarding-explainer.md` (Phase 1) + + ```bash + grep -rn "Memory injection starts on your second session in a project" src/ plugin/ + # expect 3+ matches + ``` + +2. Same for the privacy line (`Everything stays in ~/.claude-mem on this machine.`). + +3. Confirm `/how-it-works` slash reference appears in install.ts and SearchRoutes.ts; SKILL.md exists at `plugin/skills/how-it-works/SKILL.md`. + +4. Confirm WelcomeCard does NOT inline the explainer body — only the link. + +5. Confirm no surface still says "/knowledge-agent" or "/mem-search" in install.ts post-install copy. + +### Verification + +- All grep checks pass. +- Manual: read all three surfaces side by side. Each is distinctly framed (install = two paths + first-success; SessionStart = third-person status; viewer = empty/has-data with live dot). The two verbatim lines and the `/how-it-works` link are the only repeated content. + +--- + +## Phase 7 — End-to-End Smoke Test (manual) + +**Why:** The acceptance criterion in the brief is a single coherent flow. This phase walks it. + +### Steps + +1. Fresh install: + ```bash + rm -rf ~/.claude-mem + npx claude-mem install + ``` + Verify: install Next Steps shows the new "Two paths" + first-success + timing + privacy + `/how-it-works` block. + +2. Open the viewer at the printed URL. Verify: empty state shows, dot is green (connected) or red+pulsing (disconnected briefly). + +3. Open Claude Code in any project. Type a prompt that causes one Read. + - Verify in Claude Code: SessionStart context contains the new status note, NOT a "Welcome" block. Claude does not act on the bullets — at most relays them. + - Verify in viewer: card flips to has-data state, stat row populates, observation appears in the feed. + +4. End the session. Start a second Claude Code session in the same project. + - Verify: SessionStart context this time contains injected past observations (not the welcome hint, since `observationCount > 0`). + +5. Click the "How it works" link from the viewer card. Verify: it loads `/api/onboarding/explainer` markdown. + +### Verification + +All four observable beats in the acceptance criterion happen as described, in order, without any surface contradicting another on facts (timing, privacy, command names). + +--- + +## Execution Order Summary + +1. **Phase 1** (explainer + skill) — unblocks everything else by establishing the canonical content source. +2. **Phase 4** (`/api/stats` extension) — unblocks Phase 5; tiny isolated backend change, do it early. +3. **Phase 2** (SessionStart hint rewrite + default-pinning test) — independent, do in parallel with Phase 3. +4. **Phase 3** (install.ts Next Steps rewrite) — independent of Phase 2. +5. **Phase 5** (WelcomeCard rewrite) — depends on Phases 1 + 4. +6. **Phase 6** (drift audit) — runs after all copy changes land. +7. **Phase 7** (manual smoke) — final gate before commit/PR. + +Phases 2, 3, and 4 are independent and could be parallelized in three short sessions if desired. + +## Out of Scope (do not touch) + +- Splash banner / installer animation work that just shipped on this branch. +- `/learn-codebase`, `/mem-search`, `/knowledge-agent` skill internals — only how we reference them. +- New viewer pages or routes beyond `/api/onboarding/explainer` and the `firstObservationAt` field on `/api/stats`. +- Public docs in `docs/public/` — covered by the existing Mintlify deploy; only update if a doc page directly contradicts the new copy. diff --git a/plugin/hooks/hooks.json b/plugin/hooks/hooks.json index 5e9e3274..72e4d19a 100644 --- a/plugin/hooks/hooks.json +++ b/plugin/hooks/hooks.json @@ -8,7 +8,7 @@ { "type": "command", "shell": "bash", - "command": "export PATH=\"$HOME/.nvm/versions/node/v$(ls \\\"$HOME/.nvm/versions/node\\\" 2>/dev/null | sed 's/^v//' | sort -t. -k1,1n -k2,2n -k3,3n | tail -1)/bin:$HOME/.local/bin:/usr/local/bin:/opt/homebrew/bin:$PATH\"; _R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=$(ls -dt \"$HOME/.claude/plugins/cache/thedotmack/claude-mem\"/[0-9]*/ 2>/dev/null | head -1); _R=\"${_R%/}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; command -v cygpath >/dev/null 2>&1 && { _W=$(cygpath -w \"$_R\" 2>/dev/null); [ -n \"$_W\" ] && _R=\"$_W\"; }; node \"$_R/scripts/smart-install.js\"", + "command": "export PATH=\"$HOME/.nvm/versions/node/v$(ls \\\"$HOME/.nvm/versions/node\\\" 2>/dev/null | sed 's/^v//' | sort -t. -k1,1n -k2,2n -k3,3n | tail -1)/bin:$HOME/.local/bin:/usr/local/bin:/opt/homebrew/bin:$PATH\"; _R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=$(ls -dt \"$HOME/.claude/plugins/cache/thedotmack/claude-mem\"/[0-9]*/ 2>/dev/null | head -1); _R=\"${_R%/}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; command -v cygpath >/dev/null 2>&1 && { _W=$(cygpath -w \"$_R\" 2>/dev/null); [ -n \"$_W\" ] && _R=\"$_W\"; }; node \"$_R/scripts/version-check.js\"", "timeout": 300 } ] @@ -21,12 +21,6 @@ { "type": "command", "shell": "bash", - "command": "export PATH=\"$($SHELL -lc 'echo $PATH' 2>/dev/null):$PATH\"; _R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=$(ls -dt \"$HOME/.claude/plugins/cache/thedotmack/claude-mem\"/[0-9]*/ 2>/dev/null | head -1); _R=\"${_R%/}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; command -v cygpath >/dev/null 2>&1 && { _W=$(cygpath -w \"$_R\" 2>/dev/null); [ -n \"$_W\" ] && _R=\"$_W\"; }; node \"$_R/scripts/smart-install.js\"", - "timeout": 300 - }, - { - "type": "command", - "shell": "bash", "command": "export PATH=\"$($SHELL -lc 'echo $PATH' 2>/dev/null):$PATH\"; _R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=$(ls -dt \"$HOME/.claude/plugins/cache/thedotmack/claude-mem\"/[0-9]*/ 2>/dev/null | head -1); _R=\"${_R%/}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; command -v cygpath >/dev/null 2>&1 && { _W=$(cygpath -w \"$_R\" 2>/dev/null); [ -n \"$_W\" ] && _R=\"$_W\"; }; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" start; echo '{\"continue\":true,\"suppressOutput\":true}'", "timeout": 60 }, diff --git a/plugin/scripts/bun-runner.js b/plugin/scripts/bun-runner.js index a7db9d4e..b8c821d2 100644 --- a/plugin/scripts/bun-runner.js +++ b/plugin/scripts/bun-runner.js @@ -1,16 +1,4 @@ #!/usr/bin/env node -/** - * Bun Runner - Finds and executes Bun even when not in PATH - * - * This script solves the fresh install problem where: - * 1. smart-install.js installs Bun to ~/.bun/bin/bun - * 2. But Bun isn't in PATH until terminal restart - * 3. Subsequent hooks fail because they can't find `bun` - * - * Usage: node bun-runner.js