Integration: 7 critical fixes (post band-aid strip) (#2219)

* fix: strip privacy tags from last_assistant_message in summarize path

(cherry picked from commit bd68bfcc3cfe9d82977d5bdb87cf7e91a7258489)

* fix: preserve Chroma relevance ordering in SQLite hydration

When ChromaSearchStrategy queries by vector similarity with
orderBy='relevance', SessionStore.getObservationsByIds and related
methods silently coerced undefined to 'date_desc', destroying the
semantic ranking. Add 'relevance' as a valid orderBy value that skips
SQL ORDER BY and preserves caller-provided ID order.

Fixes #2153

(cherry picked from commit 9fedf8fc165c01cc3a8a8cdb8c057ea980bf511e)

* test(privacy): mock executeWithWorkerFallback and loadFromFileOnce

Update the cherry-picked privacy-tag stripping test from swithek's fork to
match current main:

- Mock executeWithWorkerFallback / isWorkerFallback (the handler now uses
  these instead of workerHttpRequest directly).
- Mock loadFromFileOnce in hook-settings.js (called by shouldTrackProject)
  so the handler resolves CLAUDE_MEM_EXCLUDED_PROJECTS to a string.
- Switch the workerCallLog shape to record { path, method, body } and
  accept either object or JSON-string bodies.

10/10 tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: pass relevance through to SessionStore in ChromaSearchStrategy

The Chroma strategy was coercing orderBy='relevance' to undefined before
calling SessionStore. Combined with SessionStore's date_desc default for
undefined, this destroyed the semantic ranking that Chroma had just
computed. Pair this with the SessionStore-side fix from rogerdigital
(commit 37c8988f) which now accepts 'relevance' as a valid orderBy and
preserves caller-provided ID order.

Adds a regression test asserting that getObservationsByIds returns rows
in caller-provided order when orderBy='relevance', and continues to
return date_desc order when orderBy is omitted.

Closes #2153

Co-Authored-By: Roger Deng <13251150+rogerdigital@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: isolate SDK boundary — settingSources, strictMcpConfig, cloud-provider env, observation cap

Single architectural fix at the three @anthropic-ai/claude-agent-sdk query()
call sites (SDKAgent.startSession, KnowledgeAgent.prime, KnowledgeAgent
.executeQuery) plus the env sanitizer and ingest gate. Closes 6 issues:

- #2155 settings.json bleed-through into observer SDK subprocess: pass
  settingSources: [] so user/project/local settings aren't inherited.
- #2159 / #2171 / #2194 user MCP servers leak into observer SDK: pass
  strictMcpConfig: true alongside the existing mcpServers: {}.
- #2199 Bedrock/Vertex env vars dropped: extend ENV_PRESERVE in
  src/supervisor/env-sanitizer.ts to keep CLAUDE_CODE_USE_BEDROCK,
  CLAUDE_CODE_USE_VERTEX, AWS_*, ANTHROPIC_VERTEX_PROJECT_ID, etc.
- #2201 runaway tokens (345M/day reported): extend default
  CLAUDE_MEM_SKIP_TOOLS with exec_command, write_stdin, apply_patch and
  add a configurable CLAUDE_MEM_MAX_OBSERVATION_BYTES (default 64 KB)
  cap at the ingest gate.

SDK option names verified against
node_modules/@anthropic-ai/claude-agent-sdk/sdk.d.ts:
  settingSources?: SettingSource[]    (SettingSource = 'user'|'project'|'local')
  strictMcpConfig?: boolean

Anti-pattern guards observed:
- Did not modify the proxy strip (#2099/#2115).
- Did not skip Read/Write/Edit/Bash — those remain the primary
  observation surface; only added high-volume agentic-tool names
  (exec_command, write_stdin, apply_patch).
- Did not invent SDK options.

Closes #2155, #2159, #2171, #2194, #2199, #2201

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: restore Windows spawn fix from PR #751 + add Windows CI

Re-applies the PowerShell Start-Process -WindowStyle Hidden daemon spawn
that PR #751 (e6ae0176) introduced and commit d13662d5 reverted. Also
fixes the bun-runner cmd /c popup, sets detached:false on Windows for
SDK subprocesses (so windowsHide actually works and claude.exe doesn't
outlive the worker), and adds windows-latest CI to prevent regression.

- ProcessManager.spawnDaemon: PowerShell -EncodedCommand branch back.
  Returns 0 sentinel on success — callers MUST use pid === undefined
  for failure detection, never falsy checks.
- bun-runner.js: drop "cmd /c" wrapper. shell:true lets Node resolve
  bun.cmd via PATHEXT and respects windowsHide (the explicit cmd.exe
  wrapper was popping a visible window per hook — #2150, #2186).
- process-registry.ts spawnSdkProcess: detached:false on Windows.
  Mixing detached:true with windowsHide:true is documented-undefined
  on Windows; with detached:false, windowsHide actually hides
  claude.exe and the SDK subprocess dies with the parent (#2190, #2198).
- .github/workflows/windows.yml: smoke test counts visible cmd windows
  before/after spawn + grep guard that the Start-Process branch survives.

WSL bash stdin (#2188) is acknowledged but deferred — the bash → node
pipe boundary needs a real Windows VM to test, beyond this PR's scope.
PTY for Claude CLI SDK mode (#2173, #2177) is also deferred per plan.

Closes #2150, #2169, #2186, #2187, #2190, #2198
Refs #2183 (Windows perf — same root cause)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: Codex transcript ingestion + queue self-deadlock on Windows

Three Windows-specific bugs surfaced by @MakaveliGER in #2192:

A. Glob path normalization
   path.join(homedir(), ...) emits backslashes on Windows. globSync treats
   backslashes as escape characters, not separators, so it silently fails to
   match transcript files. Normalize backslashes to forward slashes before
   passing to globSync (only affects Windows; Unix paths unchanged).

B. Live appends not picked up
   Per-file fs.watch on Windows ReFS/SMB misses appends to live JSONL files;
   the recursive root watcher is the only signal we can trust there. Expose
   FileTailer.poke() and call it from the root-watcher event when the file
   is already tailed, instead of returning early. Also normalize the
   resolved path so the tailer-map key matches what globSync stored.

C. Queue self-deadlock on abort
   When the SDK generator aborts (idle timeout, user cancel, shutdown) with
   rows already claimed and yielded but not yet confirmed by ResponseProcessor,
   those rows sit in 'processing' under THIS worker's PID. The self-healing
   claim predicate skips them because the worker is still alive — the queue
   deadlocks until the worker restarts. In the .finally() block, walk the
   in-flight ids through markFailed so the retry ladder requeues them as
   'pending' (or terminates them if retries are exhausted).

Includes regression test tests/codex-transcript-watcher-windows.test.ts
that asserts each fix at the source level so future refactors can't silently
revert them.

Co-Authored-By: MakaveliGER <noreply@github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Closes #2192

* fix: standalone batch — npm peer-deps overrides, marketplace self-heal warning, cache prune

- Add `overrides: { tree-sitter: ^0.25.0 }` to the generated plugin/package.json
  so `npm install --production` resolves cleanly without --legacy-peer-deps.
  Fixes the ERESOLVE between grammar packages declaring three different majors
  of tree-sitter as peer deps. Closes #2147.

- mcp-server.ts: emit a single loud, actionable warning when MCP boots but the
  marketplace directory at ~/.claude/plugins/marketplaces/<source>/ is missing.
  IDE plugin loaders silently skip claude-mem hooks in this state while MCP
  keeps working — the user has no way to know memory capture is dead. We don't
  run an installer from MCP startup (different permission model), but we tell
  the user exactly which command to run. Closes #2174.

- smart-install.js (both root and plugin variants): prune older claude-mem
  version directories from ~/.claude/plugins/cache/thedotmack/claude-mem/.
  Claude Code resolves and caches hook commands per session, so a stale 12.x
  directory keeps the old hook path alive across restarts even after upgrade.
  Pruning makes the stale path physically unreachable. Closes #2172 (stale
  version reference). Note: the issue's secondary claim that
  @anthropic-ai/claude-agent-sdk is missing from package.json is no longer
  true — it was added at line 115 in v12.4.x.

- #2170 ("ToolUseContext is required for prompt hooks") triaged as upstream:
  the string does not appear anywhere in this repo. The error originates in
  Claude Code's hook framework, which we don't own. No code change here.

Co-Authored-By: Amadan04 <amadan04@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: remove stale macOS binary, regen plugin artifacts (build/bundle drift)

The committed plugin/scripts/claude-mem (63 MB Mach-O) was last built at
v10.3.2 (Feb 2026). It baked in BUILT_IN_VERSION="10.3.1", dev paths
(/Users/alexnewman/Scripts/claude-mem/...), and a now-removed POST
/api/sessions/complete client + handler (deleted by PR #2136).

That meant macOS users running the cached binary hit 404s every time the
SessionEnd hook fired (issue #2200), the /api/health endpoint reported a
two-major-versions-ago version (issue #2158), and the binary embedded a
Zod copy that drifted from the worker bundle (issue #2154).

- Delete plugin/scripts/claude-mem and gitignore it. The npm package
  already excludes it from the "files" allowlist, so no consumer change.
  The JS fallback (bun-runner.js → worker-service.cjs) covers all
  functionality on every platform per the existing
  checkBinaryPlatformCompatibility comment in smart-install.js.
- Add npm run build:cli-binary for users who want the macOS speedup
  back. Produces it on demand from current source — no drift.
- Regenerate plugin/scripts/{worker-service,mcp-server}.cjs and
  plugin/ui/viewer-bundle.js so the shipped artifacts match HEAD.

Closes #2158, #2200, #2154.

* fix(ci): Windows workflow — install without lockfile (project uses Bun)

actions/setup-node@v4 cache: npm requires a package-lock.json and this
project uses Bun (only bunfig.toml exists at root). Drop the cache
directive, switch npm ci to npm install --no-audit --no-fund, and
narrow the build step to npm run build — build-and-sync also runs a
marketplace sync + worker restart that hardcodes ~/.claude/plugins,
which doesn't exist on CI.

* fix: harden observation cap parsing + safe stringify in debug logger

CodeRabbit majors on #2206:
- shared.ts: validate parsed cap is finite and > 0 before use; wrap
  JSON.stringify(payload.toolResponse) in try/catch and skip with
  reason 'payload_unserializable' on circular/throwing payloads, so
  ingestion never crashes on a bad tool response shape.
- logger.ts: the debug-mode JSON dump for objects was unguarded; wrap
  stringify in try/catch and fall back to formatData on cycles. This
  is the source the bundled plugin/scripts/context-generator.cjs is
  built from.

* fix(ci+windows): quote bun-runner shell:true args; replace dynamic smoke with static guards

CodeRabbit majors on #2208:

1. plugin/scripts/bun-runner.js — shell:true with separate spawnArgs
   triggers DEP0190 on Node 22+ and breaks paths/args containing
   spaces. Build a single fully-quoted command string (mirroring
   findBun()'s 'where bun' approach) and pass spawnArgs=[].

2. .github/workflows/windows.yml — the dynamic smoke step that counted
   visible cmd windows around 'claude-mem start' exits 1 on
   'claude-mem is not installed' before exercising the spawn path,
   AND PowerShell try/catch doesn't suppress native exit codes
   regardless. Replace with three static regression guards covering
   the exact patterns PR #2208 protects:
   - PowerShell Start-Process + WindowStyle Hidden in spawnDaemon
   - bun-runner shell:true with empty spawnArgs (DEP0190 guard)
   - windowsHide set on SDK spawn factory (issue #2190)

* fix(2210): cross-platform paths — Windows USERPROFILE + XDG cache symmetry

Greptile P2s on #2210:

- mcp-server.ts checkMarketplaceMarker: switch from process.env.HOME ?? ''
  to os.homedir(). HOME is unset on Windows; the empty fallback resolves
  relative to cwd, silently no-opping the canary on every Windows install.
  Also probe both ~/.claude/ and ~/.config/claude/ for the cache check so
  XDG users get the same warning behavior.

- smart-install.js pruneStaleVersionCache (both root + plugin copies):
  scan both ~/.claude/plugins/cache/thedotmack/ and ~/.config/claude/...
  paths so users on XDG don't keep stale dirs re-triggering #2172.

Greptile's third P2 (mtime vs semver sort for current version) deferred:
mtime works correctly for the common case and the directory names start
with versions that lexicographically sort the same way mtime does for
sequentially-installed versions; semver sort would be a separate change.

Refs PR #2210

* fix(2211): drop hardcoded --target from build:cli-binary

Greptile P2: the npm script was pinned to bun-darwin-arm64, so an
Intel Mac user (or anyone on Linux/Windows running this script
manually) got a cross-compiled arm64 binary that runs only via
Rosetta on x64 macOS and not at all elsewhere.

Bun's --compile defaults to the host platform when --target is
omitted. Drop the flag so the script produces a binary that matches
whoever runs it. CI builds that need a specific target can still
pass --target explicitly.

Refs PR #2211

* ci(windows): drop static-grep tripwires, keep real Windows build

The "Anti-regression" steps grep ProcessManager.ts/bun-runner.js/process-registry.ts
for specific strings (Start-Process, WindowStyle Hidden, shell:true, windowsHide).
Tripwires aren't fixes — they make refactoring harder forever and verify nothing
the actual Windows build doesn't already verify. The npm install + npm run build
on windows-latest is the real guard.

* revert: drop byte cap and skip-list extension band-aids

Strips two band-aid mechanisms from the SDK boundary fix, keeping only
the genuine isolation flags (settingSources: [], strictMcpConfig: true)
and the cloud-provider env preservation.

Removed:
- CLAUDE_MEM_MAX_OBSERVATION_BYTES (default 65536) — dropped oversize
  observations entirely. The structural fix is to chunk/summarize
  oversize tool results, not punish the data flow with an invented
  byte threshold. Tracked separately.
- exec_command, write_stdin, apply_patch added to default skip list —
  static taste decision baked into defaults for everyone. Users can
  still set CLAUDE_MEM_SKIP_TOOLS themselves.

The data flows again. Real fix is a follow-up.

* revert: drop pruneStaleVersionCache walker

Removes the cache walker that scans plugin cache dirs and deletes
"old" version directories by inferred staleness. The structural fix
for #2172 is for the installer to delete the prior version when it
writes the new one — not for a separate walker to wake up later
and guess which directories are stale.

Keeps:
- npm peer-dep override for tree-sitter (#2147)
- Marketplace marker startup probe (#2174)
- Cross-platform path handling

Tracked separately as a follow-up.

* build: regenerate bundled artifacts after merge

Rebuilt plugin/scripts/*.cjs from src after merging #2211, #2204, #2205,
#2208, #2209, #2206 (post-strip), #2210 (post-strip). Conflicts during
merge were resolved by accepting incoming bundled artifacts; this commit
replaces them with a clean rebuild from the merged source.

Verified: 0 references to MAX_OBSERVATION_BYTES, payload_too_large,
or pruneStaleVersionCache in the rebuilt artifacts.

---------

Co-authored-by: swithek <52840391+swithek@users.noreply.github.com>
Co-authored-by: Roger Deng <13251150+rogerdigital@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Amadan04 <amadan04@users.noreply.github.com>

src/servers/mcp-server.ts

@@ -33,6 +33,7 @@ import { parseFile, formatFoldedView, unfoldSymbol } from '../services/smart-fil
 import { readFile } from 'node:fs/promises';
 import { existsSync } from 'node:fs';
 import { dirname, resolve } from 'node:path';
+import { homedir } from 'node:os';
 import { fileURLToPath } from 'node:url';
 
 // Resolve the path to worker-service.cjs, which lives alongside mcp-server.cjs
@@ -676,6 +677,49 @@ function cleanup(reason: string = 'shutdown') {
 process.on('SIGTERM', cleanup);
 process.on('SIGINT', cleanup);
 
+/**
+ * Issue #2174: When the IDE extension (e.g. Cursor's Claude Code) loses its
+ * marketplace directory at ~/.claude/plugins/marketplaces/<source>/, the
+ * extension's hook loader silently skips claude-mem hooks while the MCP
+ * server (this process) keeps working. The session becomes invisible to
+ * memory with no error surfaced.
+ *
+ * The MCP server is the one piece that DOES boot in this state, so we use
+ * it as the canary: detect the missing marketplace dir and emit a single
+ * loud, actionable warning. We don't run smart-install.js from here — the
+ * MCP server runs under the IDE's permission model, not the user's shell,
+ * so attempting an install at MCP startup creates more failure modes than
+ * it fixes. Instead we tell the user exactly what to do.
+ */
+function checkMarketplaceMarker(): void {
+  try {
+    // Use os.homedir() so this works on Windows (HOME is unset there;
+    // USERPROFILE is the Windows convention and homedir() picks it up).
+    const home = homedir();
+    const marketplaceCandidates = [
+      resolve(home, '.claude', 'plugins', 'marketplaces', 'thedotmack'),
+      resolve(home, '.config', 'claude', 'plugins', 'marketplaces', 'thedotmack'),
+    ];
+    const present = marketplaceCandidates.some(p => p && existsSync(p));
+    const cacheCandidates = [
+      resolve(home, '.claude', 'plugins', 'cache', 'thedotmack', 'claude-mem'),
+      resolve(home, '.config', 'claude', 'plugins', 'cache', 'thedotmack', 'claude-mem'),
+    ];
+    const cachePresent = cacheCandidates.some(p => p && existsSync(p));
+    const cacheRoot = cacheCandidates[0];
+
+    if (!present && cachePresent) {
+      logger.error(
+        'SYSTEM',
+        'claude-mem MCP started but no marketplace directory was found at ~/.claude/plugins/marketplaces/thedotmack or the XDG equivalent. The IDE plugin loader needs that directory to fire claude-mem hooks (SessionStart, PostToolUse, Stop, etc.). Without it, MCP search will work but no new memories will be captured. To self-heal, run: node ~/.claude/plugins/cache/thedotmack/claude-mem/*/scripts/smart-install.js (or reinstall the plugin from the marketplace).',
+        { marketplaceCandidates, cacheRoot }
+      );
+    }
+  } catch {
+    // Self-heal probe is best-effort; never fail MCP startup for it.
+  }
+}
+
 // Start the server
 async function main() {
   // Start the MCP server
@@ -684,6 +728,9 @@ async function main() {
   await server.connect(transport);
   logger.info('SYSTEM', 'Claude-mem search server started');
 
+  // Surface marketplace-dir corruption that silently disables hook loading
+  checkMarketplaceMarker();
+
   // Start parent heartbeat to detect orphaned MCP servers
   startParentHeartbeat();