claude-mem/PATHFINDER-2026-04-21/06-implementation-plan.md

Pathfinder Phase 6: Implementation Plan

Date: 2026-04-22 Source: PATHFINDER-2026-04-21/05-clean-flowcharts.md Scope: 15 execution phases to land the brutal-audit cleanup. Each phase is self-contained so it can be run in a fresh chat session.

Design authority: 05-clean-flowcharts.md is the canonical design doc. This plan references it by section number (e.g., "05: 3.2" = section 3.2 of the clean-flowcharts file). When the plan and audit disagree, the plan's verified-findings take precedence — those corrections are called out explicitly in Phase 0.

---

Phase 0 — Documentation Discovery (ALREADY COMPLETED)

The design docs needed for this plan have been read and verified against the live codebase. Do not re-do this phase; cite its outputs from later phases.

Sources consulted

1. PATHFINDER-2026-04-21/05-clean-flowcharts.md — brutal audit + 12 clean flowcharts (Part 3), timer census (Part 4), deletion ledger (Part 5), execution order (Part 6), non-cull list (Part 7)
2. PATHFINDER-2026-04-21/02-duplication-report.md — 12 cross-feature duplication findings (background)
3. PATHFINDER-2026-04-21/03-unified-proposal.md — earlier consolidation targets (U1–U8)
4. Live codebase at /Users/alexnewman/.superset/worktrees/claude-mem/vivacious-teeth/src/**/*.ts

Verified-findings corrections (supersede the audit where they disagree)

These were produced by four parallel discovery subagents. Use these numbers in every downstream phase.

#	Audit claimed	Reality	Impact on plan
V1	Summary path only strips <system-reminder> (summarize.ts:66, SessionRoutes.ts:669)	Summary paths strip ZERO tags. handleSummarize (SessionRoutes.ts:491) and handleSummarizeByClaudeId (SessionRoutes.ts:669) pass last_assistant_message straight to queueSummarize with no strip.	Privacy gap is worse than audit — fix must be added to ingestSummary, not a one-line patch.
V2	Legacy handleObservations with no-strip at SessionRoutes.ts:378	handleObservations is at SessionRoutes.ts:464. It does not strip tags. handleObservationsByClaudeId at SessionRoutes.ts:560 does strip (lines 629–633).	Delete/consolidate both into ingestObservation helper.
V3	stripMemoryTagsFromJson + stripMemoryTagsFromPrompt wrappers exist	Confirmed. src/utils/tag-stripping.ts:79 and :89 both delegate to stripTagsInternal at line 48. Six sequential .replace() calls at lines 61–66.	U6 target is exact.
V4	Only 3 files call any stripMemoryTags* variant	Confirmed. SessionRoutes.ts:629, :633, :862. No call sites in summary, legacy observation, or summarize hook.	After U6, verify call-site count equals number of new ingest helpers × text fields.
V5	startUnifiedReaper at process-registry.ts:492	Does not exist. Supervisor registry (src/supervisor/process-registry.ts, 408 lines) has ProcessRegistry class + reapSession() (line 292) but no background timer. Both reapers live in the worker layer.	Phase 6 builds ReaperTick fresh in worker-service.ts; supervisor registry stays as-is.
V6	Two reapers in worker	Confirmed. startOrphanReaper (src/services/worker/ProcessRegistry.ts:508, invoked from worker-service.ts:537, 30 s). staleSessionReaperInterval (inline setInterval at worker-service.ts:547, 2 min, calls SessionManager.reapStaleSessions). Orphan reaper does not call reapStaleSessions.	Phase 6 replaces both.
V7	coerceObservationToSummary exists + non-XML early-fail + circuit breaker	Confirmed. Private fn at src/sdk/parser.ts:222. Non-XML fail at ResponseProcessor.ts:87–108. Circuit breaker at ResponseProcessor.ts:176–200 using session.consecutiveSummaryFailures.	Phase 3 deletion set is exact.
V8	500 ms poll up to 110 s in summarize hook	Confirmed. src/cli/handlers/summarize.ts:117–150. Constants: POLL_INTERVAL_MS = 500 (:24), MAX_WAIT_FOR_SUMMARY_MS = 110_000 (:25).	Phase 11 replaces with blocking endpoint.
V9	SessionRoutes has 8 endpoints	Actually 10: six under /sessions/:sessionDbId/* (:377–:382) and five under /api/sessions/* (:385–:389). /api/sessions/status is the one summary-hook polls.	Phase 11 collapses to 4; deletes are larger than audit implied.
V10	ensureWorkerRunning at every hook entry	Confirmed. Called in all 8 CLI handlers (context.ts:19, user-message.ts:35, summarize.ts:44, observation.ts, file-context.ts, file-edit.ts, session-init.ts, session-complete.ts).	Phase 1 hook-cache module lands before endpoint consolidation.
V11	SearchManager thin facade	Confirmed for @deprecated methods (queryChroma at :59, searchChromaForTimeline at :70) — but search() at :161–445 does real work (result combining, date filtering, grouping, markdown tables). File is 2069 lines.	Phase 4 keeps display-wrap, deletes deprecated + passthroughs only.
V12	27 migrations	22 private methods in MigrationRunner.runAllMigrations (lines 22–41 of src/services/sqlite/migrations/runner.ts); legacy system adds ~5 more. schema_versions table created at runner.ts:55.	Phase 9 target is "22+legacy → schema.sql + N upgrade migrations".
V13	Python sqlite3 subprocess ~120 lines	Python script embedded; invoked via execSync('python3 ...') at tests/services/sqlite/schema-repair.test.ts:62 (test file is 253 lines; production script similar).	Phase 9 deletion confirmed; move to user-facing claude-mem repair subcommand.
V14	30-s content-hash dedup window + findDuplicateObservation ~30 lines	Confirmed at src/services/sqlite/observations/store.ts:13 (DEDUP_WINDOW_MS = 30_000). findDuplicateObservation is 11 lines at :36–46. Dedup key is SHA of (memory_session_id, title, narrative) — not tool_use_id.	Phase 9 adds UNIQUE(session_id, tool_use_id) constraint and removes window; this is a new constraint, not an existing one.
V15	No chroma_synced column	Confirmed. Phase 10 must add it in a migration.	Blocks Phase 10's backfill simplification.
V16	Granular per-field Chroma docs (3–5 per obs)	Confirmed. 7 observation fields + 6 summary fields (ChromaSync.ts:125–256). formatObservationsAsDocs and formatSummariesAsDocs produce separate docs.	Phase 10 concatenates into one doc per observation/summary.
V17	getExistingChromaIds metadata scan + delete-then-add on conflict	Confirmed. getExistingChromaIds at ChromaSync.ts:479–545 pages via chroma_get_documents with include: ['metadatas']. Delete-then-add at :292–306.	Phase 10 replaces with upsert using stable IDs.
V18	5-s rescan + pendingTools map + HTTP loopback	Confirmed. src/services/transcripts/watcher.ts:124 (rescanIntervalMs ?? 5000). pendingTools in SessionState interface. observation.ts:17 loops through workerHttpRequest('/api/sessions/observations', …). Watcher calls handler directly; handler HTTPs back to worker.	Phase 7 replaces with fs.watch(parentDir, {recursive}) and direct ingestObservation(payload) call.
V19	60-s stale reset in every claimNextMessage	Confirmed. src/services/sqlite/PendingMessageStore.ts:99–145. Constant STALE_PROCESSING_THRESHOLD_MS = 60_000 at :6.	Phase 6 moves the reset to worker startup.
V20	Rate limiter 300/min	Confirmed at src/services/worker/http/middleware.ts:45–79. Constants at :49–50. Keyed by IP, normalizes ::ffff:127.0.0.1.	Phase 14 deletes.

Allowed APIs (what the refactor may rely on)

Copy from these exact sources; do not invent.

• bun:sqlite — Database, db.prepare(sql), db.run, db.transaction(fn). Unique constraint: CREATE TABLE x (... UNIQUE(a,b)). Conflict clause: INSERT ... ON CONFLICT DO NOTHING or ON CONFLICT (a,b) DO UPDATE SET .... (Used everywhere under src/services/sqlite/.)
• Express 4 — app.get/post, router.use(middleware), req.body, res.json, res.sendFile, SSE via res.write('event: …\ndata: …\n\n'). (See BaseRouteHandler.ts, SSEBroadcaster.ts.)
• Zod — z.object({...}), schema.safeParse(body), result.success ? result.data : result.error.flatten(). (Not yet a dep; Phase 12 adds zod via npm; already shipped transitively via @anthropic-ai/sdk — confirm before landing.)
• Node fs.watch — fs.watch(dir, { recursive: true }, (event, filename) => …). On macOS + Linux recursive is supported; Windows is too. New files in the watched directory fire rename events. (Replaces the 5-s rescan timer.)
• Claude Agent SDK @anthropic-ai/claude-agent-sdk — existing usage in src/services/worker/SDKAgent.ts. Agent contract requires <summary> OR <skip_summary/>; see src/sdk/prompts.ts for the exact instruction text.

Anti-patterns to prohibit (cite in every phase)

A. Inventing APIs — never add a method to a class because it "should exist". Grep the class first. B. Polling where events exist — setInterval + HTTP poll replaced by blocking endpoint or SSE. C. Silent fallbacks — Chroma failure returns 503, not dropped-query-text search. Parser failure marks pending_messages FAILED, not coerced summary. D. Facades that pass through — if a method body is return this.other.method(args), delete it; call this.other directly. E. Two code paths for the same data — if transcript watcher and CLI handler both ingest observations, they call the same helper. No duplicate tag-strip logic.

---

Phase 1 — One stripMemoryTags + close summary privacy gap

Outcome: A single public stripMemoryTags(text: string): string. Every text-ingress call-site switches to it. Summary paths strip tags (closes P1 security bug).

Context this phase needs

• 05-clean-flowcharts.md section 3.2 (privacy-tag-filtering clean flowchart)
• Verified-findings V1, V2, V3, V4
• src/utils/tag-stripping.ts:48–91 — existing wrappers

Tasks

1. Rewrite src/utils/tag-stripping.ts to export:

   const MEMORY_TAGS = ['private','claude-mem-context','system_instruction','system-instruction','persisted-output','system-reminder'] as const;
   const STRIP_REGEX = new RegExp(`<(${MEMORY_TAGS.join('|')})\\b[^>]*>[\\s\\S]*?<\\/\\1>`, 'g');
   export function stripMemoryTags(text: string): string { /* one pass; ReDoS guard if match count > 100 */ }
   

   Delete stripMemoryTagsFromPrompt, stripMemoryTagsFromJson, stripTagsInternal, SYSTEM_REMINDER_REGEX. Keep the length/timing guards from the existing file if they're there today.
2. Fix every call site to use stripMemoryTags:
   • SessionRoutes.ts:629,633 (was stripMemoryTagsFromJson): call on JSON.stringify(tool_input) and JSON.stringify(tool_response) — same shape, new name.
   • SessionRoutes.ts:862 (was stripMemoryTagsFromPrompt): unchanged signature.
   • Add in SessionRoutes.ts:464 (legacy handleObservations): strip tool_input and tool_response before queueObservation.
   • Add in SessionRoutes.ts:491 (handleSummarize): strip last_assistant_message before queueSummarize.
   • Add in SessionRoutes.ts:669 (handleSummarizeByClaudeId): same.
3. Update the test tests/utils/tag-stripping.test.ts (if present) to cover the merged function; delete tests for the removed wrappers.

Verification

• grep -r "stripMemoryTagsFromJson\|stripMemoryTagsFromPrompt\|stripTagsInternal" src/ → zero hits.
• grep -c "stripMemoryTags(" src/ ≥ 5 (new call sites: 3 existing + 3 new summary/legacy paths).
• Regression test: insert <private>secret</private> into a summary via /sessions/:id/summarize; assert session_summaries.last_assistant_message contains no <private> or secret.
• npm run build-and-sync succeeds.

Anti-pattern guards

• A: Don't add a stripMemoryTagsV2 wrapper — rename in place.
• D: Don't leave the old function names as re-exports "for safety" — delete.

Blast radius

Edits: 2 files (tag-stripping.ts, SessionRoutes.ts). No schema changes.

---

Phase 2 — Unified ingest helpers

Outcome: Three helpers that every ingest point calls. No HTTP loopback inside the worker process.

Context this phase needs

• 05-clean-flowcharts.md section 3.1 (lifecycle-hooks clean), Part 2 Decision D1
• Verified-findings V2, V18
• Phase 1 MUST be done first.

Tasks

1. Create src/services/worker/ingest/index.ts exporting:

   export function ingestObservation(payload: IngestObservationPayload): Promise<IngestResult>;
   export function ingestPrompt(payload: IngestPromptPayload): Promise<IngestResult>;
   export function ingestSummary(payload: IngestSummaryPayload): Promise<IngestResult>;
   

   Each helper: (a) calls stripMemoryTags on user-facing text fields, (b) runs privacy / project-exclusion validation (move logic from SessionRoutes.handleObservationsByClaudeId:614–621 and PrivacyCheckValidator.ts), (c) INSERTs into pending_messages. Returns { skipped: boolean, id?: number, reason?: string }.
2. Rewire SessionRoutes.ts:464 (handleObservations), :560 (handleObservationsByClaudeId), :491 + :669 (summarize), :862 (handleSessionInitByClaudeId → ingestPrompt) to call the helpers. Route handler's job shrinks to body parsing + response serialization.
3. Rewire src/cli/handlers/observation.ts to call ingestObservation directly when the worker is the current process — but since hooks run in CLI, they still HTTP to the worker. The key change: the worker side of the route talks to the helper, no more inline logic.
4. Rewire src/services/transcripts/watcher.ts to call ingestObservation(payload) directly (no workerHttpRequest from inside the worker). Delete the inner HTTP call from the transcript path.

Verification

• grep -n "stripMemoryTags" src/services/worker/ → only inside ingest/index.ts.
• grep -n "queueObservation\|queueSummarize" src/services/worker/http/routes/SessionRoutes.ts → zero (handlers use ingest helpers).
• Unit tests for each helper: tag stripping, privacy validation, project exclusion, INSERT behaviour, idempotent returns for dup.
• Integration: run full hook cycle via npm run build-and-sync + trigger SessionStart + PostToolUse; observe pending_messages row.

Anti-pattern guards

• E: Don't leave behind handleObservations and handleObservationsByClaudeId with slightly different logic. One helper, both handlers call it.
• A: No IngestService class unless two existing classes already share state. A module with three functions is enough.

Blast radius

Files touched: SessionRoutes.ts, new ingest/*, watcher.ts, PrivacyCheckValidator.ts (may collapse into helper). No schema changes.

---

Phase 3 — Unify parser; delete coerce + circuit breaker

Outcome: One parseAgentXml(text, {requireSummary}). coerceObservationToSummary, consecutive-failure counter, and non-XML early-fail branch are gone. RestartGuard handles repeated failures.

Context this phase needs

• 05-clean-flowcharts.md section 3.7, Part 2 Decision D5
• Verified-findings V7
• src/sdk/parser.ts, src/services/worker/agents/ResponseProcessor.ts:87–200, src/services/worker/RestartGuard.ts
• src/sdk/prompts.ts — agent instructions must already state "return <summary> or <skip_summary/>". If not, update the prompt in this phase.

Tasks

1. Replace parser.ts with:

   export interface ParsedAgentOutput {
     observations: ParsedObservation[];
     summary: ParsedSummary | null;
     skipSummary: boolean;
   }
   export interface ParseResult {
     valid: boolean;
     data?: ParsedAgentOutput;
     reason?: 'no_xml' | 'missing_summary' | 'malformed';
   }
   export function parseAgentXml(text: string, opts: { requireSummary: boolean }): ParseResult;
   

   Delete parseObservations and parseSummary exports; keep them as private helpers only if the call sites merge into one. Delete coerceObservationToSummary outright.
2. Update ResponseProcessor.ts:
   • Replace the parse path with a single parseAgentXml(text, {requireSummary: session.expectsSummary}).
   • On valid:false: call session.recordFailure(result.reason) → mark pending_messages FAILED → let RestartGuard decide. Delete lines :87–108 (non-XML early-fail), lines :176–200 (consecutiveSummaryFailures counter + circuit).
   • Remove the consecutiveSummaryFailures field from ActiveSession.
3. Update sdk/prompts.ts if needed so the agent contract is explicit: on work → one or more <observation> then exactly one <summary>; on no work → <skip_summary/>.

Verification

• grep -n "coerceObservationToSummary\|consecutiveSummaryFailures" src/ → zero hits.
• grep -n "parseObservations\|parseSummary" src/ | grep -v parser.ts → zero (callers use parseAgentXml).
• Test: inject garbage-text agent output; assert pending_messages.status = 'failed' and no summary row written.
• Test: inject valid <observation> without <summary> when requireSummary=true; assert valid:false, reason:'missing_summary'.
• RestartGuard still trips after N consecutive failures (unchanged count).

Anti-pattern guards

• C: Don't coerce "close enough" to <summary>. Fail fast.
• A: No new ParserValidator class. Pure function returns a result object.

Blast radius

Files: parser.ts, ResponseProcessor.ts, possibly prompts.ts, ActiveSession (remove counter field). No schema changes.

---

Phase 4 — Delete SearchManager pass-throughs

Outcome: HTTP route → SearchOrchestrator directly. SearchManager shrinks to the display-wrap only.

Context this phase needs

• 05-clean-flowcharts.md section 3.6
• Verified-finding V11
• src/services/worker/SearchManager.ts (2069 lines) and src/services/worker/http/routes/SearchRoutes.ts

Tasks

1. Route rewire: SearchRoutes.ts handlers call SearchOrchestrator.search(params) directly for structured results, then renderSearchResults(results, format) (new small helper extracted from current SearchManager) for markdown.
2. Delete from SearchManager.ts:
   • queryChroma (:59, @deprecated) — delete all call sites first (grep).
   • searchChromaForTimeline (:70) — delete.
   • Any method whose body is return this.orchestrator.foo(...) with no other work.
3. Keep the result-combining / grouping / markdown-table code in SearchManager.search() as a renderSearchResults(results, opts) module. This is real work (V11). Put it in src/services/worker/search/ResultRenderer.ts if not already there.
4. Delete filterByRecency default 90-day filter. Callers pass dateRange explicitly.

Verification

• grep -n "class SearchManager" src/ → file either deleted or reduced to < 200 lines of display logic.
• grep -n "queryChroma\|searchChromaForTimeline" src/ → zero.
• grep -n "filterByRecency" src/ → zero.
• Integration: curl '/api/search?q=test&project=cm&format=markdown' and format=json — both return expected shapes.

Anti-pattern guards

• D: A method that forwards must die.
• C: If Chroma is disabled and q is set, return 503 with error: 'chroma_unavailable' — don't silently run a SQLite fallback.

Blast radius

SearchManager.ts, SearchRoutes.ts, new ResultRenderer.ts. No schema changes.

---

Phase 5 — Delete worker ProcessRegistry facade

Outcome: Worker talks to src/supervisor/process-registry.ts directly. src/services/worker/ProcessRegistry.ts becomes a small module of free functions for spawning and SIGTERM→SIGKILL escalation (not a registry).

Context this phase needs

• 05-clean-flowcharts.md section 3.8, Part 2 Decision D3
• Verified-findings V5, V6
• src/services/worker/ProcessRegistry.ts (527 lines), src/supervisor/process-registry.ts (408 lines), src/services/worker-service.ts (uses both)

Tasks

1. Audit worker/ProcessRegistry.ts exports and rehome:
   • registerProcess, unregisterProcess, getProcessBySession, getActiveCount, waitForSlot, getActiveProcesses, reapOrphanedProcesses → these wrap the supervisor's registry. Delete the worker copies; callers switch to getSupervisor().getRegistry().foo(…) (already what they ultimately hit).
   • ensureProcessExit (:185, SIGTERM→SIGKILL escalation) → keep as a free function in a new src/services/worker/process-control.ts. Inline the 5-s wait + SIGKILL. Remove the ladder-framework packaging.
   • createPidCapturingSpawn (:393) → move to process-control.ts.
   • startOrphanReaper (:508) → delete in Phase 6 (replaced by ReaperTick).
2. Delete src/services/worker/ProcessRegistry.ts when it's empty.
3. Update all imports (grep for from.*worker/ProcessRegistry and re-point).

Verification

• test -f src/services/worker/ProcessRegistry.ts → false.
• grep -rn "worker/ProcessRegistry" src/ → zero.
• All worker + tests still compile: npx tsc --noEmit.
• Manual test: start worker, spawn a summarize subprocess, SIGTERM it → observe SIGKILL after 5 s.

Anti-pattern guards

• D: Do not add a "compatibility shim" that re-exports the deleted symbols.
• A: ensureProcessExit is five lines — don't build a class for it.

Blast radius

Big import fan-out. Compile-time breakage until all imports are fixed. Runtime: identical behavior (supervisor registry was always the backing store).

---

Phase 6 — ReaperTick: single 30-s timer with three checks

Outcome: One setInterval(30_000) in worker-service.ts. Three skippable checks: prune dead PIDs (every tick), kill hung generators (every 4 ticks), delete abandoned sessions (every 4 ticks). The per-claim 60-s stale reset runs once at boot instead.

Context this phase needs

• 05-clean-flowcharts.md section 3.8 subgraph OneReaper, Part 4 timer census
• Verified-findings V6, V19
• Phase 5 MUST be done.

Tasks

1. Create src/services/worker/reaper.ts:

   export function startReaperTick(deps: {
     processRegistry: ProcessRegistry;
     sessionManager: SessionManager;
     pendingStore: PendingMessageStore;
     thresholds?: { generatorIdleMs?: number; sessionIdleMs?: number };
   }): { stop(): void };
   

   Internally: tick counter, reapDeadPids() every tick, reapHungGenerators() + reapAbandonedSessions() every 4 ticks. Thresholds: generatorIdleMs=5*60_000, sessionIdleMs=15*60_000.
2. Delete startOrphanReaper (ProcessRegistry.ts:508) and staleSessionReaperInterval (worker-service.ts:547). Delete reapOrphanedProcesses, killSystemOrphans, killIdleDaemonChildren as separate functions; fold their bodies into reapDeadPids.
3. Move PendingMessageStore.claimNextMessage stale reset from inside the claim (lines :99–145) into a new PendingMessageStore.recoverStuckProcessing() method called once at worker boot in worker-service.ts after the DB is ready. The claim becomes a clean SELECT ... LIMIT 1 FOR UPDATE-equivalent transaction.
4. Update worker-service.ts shutdown path to stop() the ReaperTick before orphan reaper (it's the same thing now).

Verification

• grep -n "setInterval" src/services/worker*/ → exactly one call (inside reaper.ts).
• grep -n "staleSessionReaperInterval\|startOrphanReaper" src/ → zero.
• grep -A3 "STALE_PROCESSING_THRESHOLD_MS" src/services/sqlite/PendingMessageStore.ts → threshold used only in recoverStuckProcessing.
• Integration test: kill the SDK subprocess for a running session; within 30 s the ProcessRegistry has unregistered and SessionManager entry is gone.
• Boot recovery test: insert pending_messages row with status=processing, started_processing_at_epoch=epoch-2hr; start worker; assert row flipped back to pending within boot.

Anti-pattern guards

• B: No polling loops. claimNextMessage must not do self-healing on each call.
• A: No Reaper class unless a second state ever has to live there. Start as a function.

Blast radius

Worker lifecycle + SQLite claim path. Risk: reaper timing regression. Mitigation: keep the three thresholds identical to today.

---

Phase 7 — Transcript watcher cleanup

Outcome: fs.watch(parent_dir, {recursive: true}) instead of 5-s rescan. No pendingTools state map (match by tool_use_id at line boundary). Direct ingestObservation call; no HTTP loopback from inside worker.

Context this phase needs

• 05-clean-flowcharts.md section 3.12
• Verified-finding V18
• Phases 2, 5, 6 MUST be done.

Tasks

1. Rewrite src/services/transcripts/watcher.ts:
   • Replace periodic rescan (setInterval(… 5000)) with fs.watch(parentDir, { recursive: true }, onFileEvent). Handle rename events to add new files, change events to tail existing ones.
   • Delete rescanIntervalMs config option and the watcher-internal timer.
2. Rewrite src/services/transcripts/processor.ts:
   • Remove pendingTools: Map<string, {name?, input?}> from SessionState.
   • When a JSONL line is a tool_use → enqueue into a per-file map keyed by tool_use_id. When a later line is a tool_result with the same tool_use_id, emit one IngestObservationPayload and drop the entry. If a tool_use has no tool_result after N lines (say, 10 MB of JSONL read), timeout-log and drop.
3. Replace HTTP loopback with import { ingestObservation } from '…/worker/ingest' and direct call.
4. Project-exclusion: let ingestObservation handle it; remove the re-check in the transcript processor.

Verification

• grep -n "setInterval" src/services/transcripts/ → zero.
• grep -n "pendingTools" src/ → zero.
• grep -n "workerHttpRequest" src/services/transcripts/ src/cli/handlers/observation.ts → count ≥ 0 (CLI handler can still HTTP the worker; only the in-process loopback is forbidden).
• Integration: drop a new Cursor transcript file into the watched dir; within 1 s a pending_messages row appears.

Anti-pattern guards

• B: No fallback polling "in case fs.watch misses an event". Parent-recursive watch is the contract.
• E: The transcript ingest path and the hook ingest path both call ingestObservation. One function, two callers.

Blast radius

Transcript watcher only. Kept user-facing: Cursor, OpenCode, Gemini-CLI JSONL ingest still works.

---

Phase 8 — Unified renderObservations(obs, strategy)

Outcome: One traversal, four strategy configs. AgentFormatter, HumanFormatter, ResultFormatter, and CorpusRenderer become strategy definitions that plug into the single renderer.

Context this phase needs

• 05-clean-flowcharts.md section 3.5 (context-injection) + Part 2 Decision D4
• Files: src/services/context/formatters/{AgentFormatter,HumanFormatter}.ts, src/services/worker/search/ResultFormatter.ts, src/services/worker/knowledge/CorpusRenderer.ts, all section renderers under src/services/context/sections/

Tasks

1. Design the renderer contract in src/services/rendering/renderObservations.ts:

   export interface RenderStrategy {
     name: 'agent' | 'human' | 'search' | 'corpus';
     columns: Array<'title'|'narrative'|'facts'|'file'|'date'|'session'|'tokens'>;
     density: 'compact' | 'normal' | 'verbose';
     grouping?: 'none' | 'by-day' | 'by-file' | 'by-session';
     colorize?: boolean; // terminal ANSI
     tokenBudget?: number;
   }
   export function renderObservations(obs: Observation[], strategy: RenderStrategy): string;
   

2. Replace each of the four formatters with a RenderStrategy object (e.g., AgentContextStrategy, HumanContextStrategy, SearchResultStrategy, CorpusDetailStrategy). The strategies live in their respective feature folders; the renderer is shared.
3. Move one-off logic (ANSI coloring, token budgeting, day-grouping) from the four formatters into the renderer, gated by strategy flags.
4. Keep mode filtering + section ordering in the builder (ContextBuilder) — only the final render step unifies.

Verification

• grep -rn "formatObservation\|renderObservation" src/ | wc -l — one shared renderer, four strategy files.
• Snapshot tests: for each strategy, feed the same fixture Observation[] and assert output is byte-equal to the old formatter's output.
• npm run build-and-sync + SessionStart injects a context block identical to pre-refactor bytes (modulo strategy-flagged differences).

Anti-pattern guards

• E: No "almost the same" paths remain. All four formatters end up as thin export const FooStrategy: RenderStrategy = … files.
• A: No RendererFactory. The renderer is a pure function.

Blast radius

Pure code reorganization, lowest risk. Snapshot tests are the safety net.

---

Phase 9 — SQLite consolidation

Outcome: Fresh DBs use schema.sql (current state). Upgrade-only migrations run for old DBs. UNIQUE(session_id, tool_use_id) added. 30-s content-hash dedup window removed. Python repair script gone; user-facing claude-mem repair command added.

Context this phase needs

• 05-clean-flowcharts.md section 3.3, Part 5 ledger rows for SQLite
• Verified-findings V12, V13, V14
• src/services/sqlite/migrations/runner.ts, src/services/sqlite/observations/store.ts, tests/services/sqlite/schema-repair.test.ts

Tasks

1. Add observations.tool_use_id column in a new migration (if not already there — grep the schema). Add UNIQUE(session_id, tool_use_id) constraint. For observations without a tool_use_id (legacy rows), set a synthetic value like legacy:<id> so the UNIQUE doesn't collide.
2. Rewrite observations/store.ts:
   • Use INSERT ... ON CONFLICT (session_id, tool_use_id) DO NOTHING RETURNING id.
   • On conflict, re-SELECT the existing row and return its id. Idempotent.
   • Delete DEDUP_WINDOW_MS, findDuplicateObservation, and the content-hash dedup query. Keep the content_hash column — it's useful for cross-machine dedup analytics; just don't use it as a dedup gate.
3. Create src/services/sqlite/schema.sql with the current schema. On fresh DB, run schema.sql then write schema_versions row at current version. On existing DB, skip schema.sql and run only migrations with version > max(schema_versions.version).
4. Delete the Python repair path (execSync('python3 …')). Add a new CLI subcommand claude-mem repair that runs the Python script on demand — this is for users who hit corruption from v<X. Document in a new docs/public/troubleshooting/repair.mdx page.
5. Consolidate migration boilerplate. 22+ migrations with CREATE TABLE IF NOT EXISTS patterns become: schema.sql covers everything; remaining upgrade migrations only do ALTER TABLE / CREATE INDEX IF NOT EXISTS / data migrations.

Verification

• Fresh-install test: delete ~/.claude-mem/claude-mem.db; start worker; assert schema_versions.version = N and all expected tables exist.
• Upgrade test: start worker on an old DB from v6.0; assert all migrations run and the final schema matches schema.sql.
• Dup test: insert two observations rows with the same (session_id, tool_use_id); assert second INSERT returns the first row's id and no duplicate row exists.
• grep -n "execSync.*python" src/ → zero.
• claude-mem repair command executes without error on a known-corrupt DB fixture.

Anti-pattern guards

• A: No "schema migration framework". bun:sqlite + a schema_versions table + a list of migration functions is enough.
• E: Don't keep both content-hash dedup and UNIQUE(session_id, tool_use_id) as two gates. Pick one (the constraint).

Blast radius

Highest-risk migration in the plan. Requires backfill of tool_use_id for rows that don't have it. Run in a staged release with the claude-mem repair fallback.

---

Phase 10 — Chroma rewrite

Outcome: One doc per observation (title + narrative + facts concatenated). Stable ID obs:<sqlite_rowid>. Upsert instead of delete-then-add. chroma_synced boolean column on observations; backfill only rows where the flag is false. Full-project scan on boot deleted.

Context this phase needs

• 05-clean-flowcharts.md section 3.4
• Verified-findings V15, V16, V17
• src/services/sync/ChromaSync.ts:125–545
• Phase 9 MUST be done (so chroma_synced migration can land alongside).

Tasks

1. Migration: add chroma_synced INTEGER DEFAULT 0 column to observations and session_summaries.
2. Rewrite ChromaSync.formatObservationAsDoc: one doc per observation. Text = title + "\n\n" + narrative + "\n\n" + facts.join("\n"). ID = obs:${sqliteRowId}. Metadata keeps project, session_id, timestamp, type. Same for summaries (one doc, stable ID).
3. Replace chromaSync.syncObservation write path: chroma_mcp.upsert(id, text, metadata). On success: UPDATE observations SET chroma_synced=1 WHERE id=?. On failure: logger.warn, leave flag 0.
4. Replace ensureBackfilled + runBackfillPipeline + getExistingChromaIds with a simple backfillUnsynced(limit=1000) called once at boot. Query: SELECT id FROM observations WHERE chroma_synced=0 LIMIT 1000. For each: format → upsert → mark.
5. Delete backfillAllProjects (static), ensureBackfilled, runBackfillPipeline, getExistingChromaIds, formatObservationsAsDocs, formatSummariesAsDocs (multi-doc), and the delete-then-add conflict handler.

Verification

• Chroma index contains one doc per observation (not 7). Query Chroma directly: chroma_count_documents(collection) = SELECT COUNT(*) FROM observations WHERE chroma_synced=1.
• Idempotent re-sync: call syncObservation twice with same ID; assert no conflict, one doc.
• Boot with Chroma down: observations sync'd to SQLite normally, chroma_synced=0. Start Chroma, restart worker: those rows upserted within boot.
• grep -n "backfillAllProjects\|ensureBackfilled\|getExistingChromaIds" src/ → zero.

Anti-pattern guards

• C: On Chroma failure at write time, do not throw — leave flag 0 and move on. The backfill path covers recovery.
• A: No ChromaBackfillScheduler. One function, called at boot, done.

Blast radius

Chroma index regenerates under the new doc shape. Users see the old index until the first boot-time backfill completes (may take minutes on large corpora).

---

Phase 11 — Endpoint consolidation

Outcome: 10 session endpoints → 4. /api/session/start returns context + semantic in one call. /api/session/end blocks until summary written or 110-s timeout (no hook-side polling). /api/context/inject + /api/context/semantic deleted or folded.

Context this phase needs

• 05-clean-flowcharts.md section 3.1, section 3.9 (Routes inventory), Part 2 Decision D6
• Verified-findings V8, V9, V10
• src/services/worker/http/routes/SessionRoutes.ts, src/services/worker/http/routes/SearchRoutes.ts, src/cli/handlers/{context,user-message,summarize,session-complete}.ts

Tasks

1. New endpoints (4 total):
   • POST /api/session/start — body: {project, claudeSessionId}. Returns {sessionDbId, contextMarkdown, semanticMarkdown}. Internally: calls ContextBuilder.generateContext + SearchOrchestrator.search.
   • POST /api/session/prompt — body: {sessionDbId, prompt}. Returns {promptId}.
   • POST /api/session/observation — body: {sessionDbId, tool_use_id, name, input, output}. Returns {observationId|null, skipped}.
   • POST /api/session/end — body: {sessionDbId, last_assistant_message}. Blocks until the queue is drained and the summary row is written (or 110-s timeout). Returns {summaryId|null}.
2. Blocking /api/session/end: implement via a per-session Deferred<SummaryResult>. When ResponseProcessor writes the summary row, resolve the deferred. Route handler awaits the promise with a 110-s race.
3. Delete the old 10 endpoints under /sessions/:sessionDbId/* and /api/sessions/* after all hook-side callers are switched. Also delete /api/context/inject and /api/context/semantic.
4. Rewrite hook handlers (context.ts, user-message.ts, summarize.ts, session-complete.ts) to use the 4 new endpoints. Delete the 500-ms polling loop in summarize.ts:117–150.
5. Hook-side ensureWorkerRunning cache: create src/hooks/worker-cache.ts that caches alive=true in module scope for the hook process. First call spawns/HTTPs /health; subsequent calls skip. Switch all 8 handlers to import from this module.

Verification

• grep -n "router\.\(get\|post\|delete\)" src/services/worker/http/routes/SessionRoutes.ts → 4 routes.
• grep -n "/api/context/inject\|/api/context/semantic" src/ → zero.
• grep -n "POLL_INTERVAL_MS\|MAX_WAIT_FOR_SUMMARY_MS" src/cli/handlers/ → zero.
• Integration: run a full session lifecycle; assert Stop hook returns within ~110 s (or earlier) with a summaryId, and no /status polling requests hit the worker.
• Perf: SessionStart latency ≤ previous latency (one request vs two).

Anti-pattern guards

• B: No polling. Blocking + timeout replaces it.
• D: /api/session/start must not be a facade over /api/context/inject; the old endpoints are deleted.

Blast radius

Hook ↔ worker HTTP contract changes. Needs coordinated plugin rebuild (npm run build-and-sync). Old hooks calling old endpoints will 404 — land after a version bump.

---

Phase 12 — Zod validator middleware

Outcome: Per-route Zod schema + one validateBody(schema) middleware. Per-route hand-rolled validation gone.

Context this phase needs

• 05-clean-flowcharts.md section 3.9
• src/services/worker/http/routes/*.ts (8 files with inline validation)

Tasks

1. Add zod to package.json dependencies (confirm not already present; if it is, skip).
2. Create src/services/worker/http/middleware/validateBody.ts:

   export function validateBody<T>(schema: z.ZodType<T>): RequestHandler { … }
   

   On parse failure: res.status(400).json({ error: 'validation_failed', fields: result.error.flatten() }).
3. Per-route schemas in a parallel schemas/ directory (or inline at top of each route file). One z.object({…}) per endpoint.
4. Delete per-route boilerplate: manual typeof x !== 'string' checks, if (!body.foo) return res.status(400)….

Verification

• grep -n "res.status(400)" src/services/worker/http/routes/ | wc -l significantly reduced (only routes that return 400 for domain reasons, not shape validation).
• Error-shape tests: each endpoint, with invalid body, returns {error, message, code, fields}.
• No behavioral regression on happy path (snapshot test of responses).

Anti-pattern guards

• A: Don't invent ZodUtil.assertBody — use safeParse directly.
• E: Single middleware, not one per route.

Blast radius

HTTP error shape might change slightly (field names in 400s). Client (viewer UI) must tolerate fields key.

---

Phase 13 — KnowledgeAgent simplification

Outcome: No session_id persistence in corpus.json. No prime endpoint. No auto-reprime regex. build IS prime; every query loads the corpus fresh as system prompt.

Context this phase needs

• 05-clean-flowcharts.md section 3.11
• src/services/worker/knowledge/KnowledgeAgent.ts, CorpusStore.ts, CorpusBuilder.ts, corresponding routes in CorpusRoutes.ts

Tasks

1. Delete KnowledgeAgent.prime and the reprime endpoint. Update the OpenAPI/route table to drop them.
2. Simplify CorpusStore: corpus JSON contains {name, filters, renderedCorpus, generatedAt}. No session_id.
3. Rewrite KnowledgeAgent.query to always pass systemPrompt = renderedCorpus to the SDK. Claude prompt-caching reduces cost when the same corpus is queried repeatedly within the 5-min TTL.
4. Delete the session-expiration regex match and auto-reprime path.

Verification

• grep -n "session_id" src/services/worker/knowledge/ → zero.
• grep -n "reprime\|auto.*reprime" src/ → zero.
• Cost test: query the same corpus 3× within 5 min; assert cache hits (the SDK returns cache_read_input_tokens > 0).
• POST /api/corpus/:name/rebuild still works; POST /api/corpus/:name/prime returns 404.

Anti-pattern guards

• C: Don't try to "detect session expiration". Always pass fresh system prompt; let the SDK cache decide.

Blast radius

Corpus JSON format changes (drops session_id). Existing corpora still load (extra field ignored or migrated on read).

---

Phase 14 — HTTP cleanup

Outcome: Rate limiter deleted. Static file reads cached at boot.

Context this phase needs

• 05-clean-flowcharts.md section 3.9
• Verified-finding V20
• src/services/worker/http/middleware.ts:45–79, ViewerRoutes.ts

Tasks

1. Delete src/services/worker/http/middleware.ts:45–79 (the rate limiter) and its registration in Middleware.ts.
2. Cache viewer.html and /api/instructions content in memory at boot; serve from Buffer instead of fs.readFile.
3. Delete the legacy SessionRoutes.handleObservations no-privacy-strip endpoint (already handled in Phase 2 if the route is rewired; this is the cleanup pass).

Verification

• grep -n "RATE_LIMIT_WINDOW_MS\|RATE_LIMIT_MAX_REQUESTS" src/ → zero.
• Boot time: viewer.html hits don't cause fs.readFile calls (measure with lsof or a log statement).

Anti-pattern guards

• B: Don't re-introduce the rate limiter as a "config flag". Localhost trust model is explicit.

Blast radius

Minimal. The rate limiter was theater on a localhost server.

---

Phase 15 — Final verification

Outcome: Whole system behaves per the clean flowcharts. Timer census reads 1 repeating timer. No polling loops. No silent fallbacks. Deleted-lines counter ≥ 2500 net.

Tasks

1. Run the timer census:

   grep -rn "setInterval\|setTimeout.*recursive\|setTimeout.*repeat" src/ | grep -v test
   

   Expected: one setInterval in reaper.ts; one per-session idle timeout; one EventSource reconnect (UI); no others. Compare against 05-clean-flowcharts.md Part 4.
2. Anti-pattern grep pass:
   • grep -rn "coerceObservationToSummary\|consecutiveSummaryFailures\|DEDUP_WINDOW_MS\|STALE_PROCESSING_THRESHOLD_MS.*claimNextMessage\|backfillAllProjects\|getExistingChromaIds\|stripMemoryTagsFromJson\|stripMemoryTagsFromPrompt\|POLL_INTERVAL_MS" src/ → zero matches.
   • grep -rn "res.status(503)" src/ includes chroma_unavailable path (positive check).
3. Deleted-lines count: git diff main --stat | tail -1 — compare against the audit's Part 5 estimate (~2500 net).
4. Run full test suite: npm test.
5. Run plugin end-to-end: npm run build-and-sync → trigger all 5 lifecycle hooks in a real Claude Code session → verify SSE events, viewer UI renders, search works, corpus builds + queries, transcript watcher picks up a synthetic Cursor log.
6. Document: update docs/public/architecture.mdx (or equivalent) to point at 05-clean-flowcharts.md as the canonical architecture doc.

Verification

• Timer census matches 05-clean-flowcharts.md Part 4 "after" column.
• All grep anti-pattern checks return zero matches.
• Full test suite green.
• End-to-end plugin test passes.

---

Phase dependency graph

P1 ─┐
    ├─> P2 ─┬─> P3
    │      ├─> P6 ─> P7
    │      └─> P11
    │
P4  (independent)
P5  ──> P6 (already sequenced above)
P8  (independent — can run anytime)
P9  ──> P10
P11 ──> P12 (Zod lands after endpoint shape is final)
P13 (independent)
P14 (after P11 so legacy route delete is clean)
P15 gates merge.

Parallelizable tracks: (P1→P2→P3), (P4), (P5→P6→P7), (P8), (P9→P10), (P13). Merge order: P1,P2,P3,P4,P5,P6,P7,P8,P9,P10,P11,P12,P13,P14,P15.

---

Estimated effort

Per 05-clean-flowcharts.md Part 6: ~18 engineer-days for full clean-through. Phase 1 alone closes the P1 security gap (<1 day).

Success criteria

• One setInterval in the worker codebase.
• Zero polling loops on the hook side.
• 40 bullshit items from 05-clean-flowcharts.md Part 1 all deleted (verified by grep).
• All 12 user-facing features from Pathfinder Phase 0 still work.
• Net LOC deleted ≥ 1800.