Files
claude-mem/src/services/worker/DatabaseManager.ts
T
Alex Newman 40daf8f3fa feat: replace WASM embeddings with persistent chroma-mcp MCP connection (#1176)
* feat: replace WASM embeddings with persistent chroma-mcp MCP connection

Replace ChromaServerManager (npx chroma run + chromadb npm + ONNX/WASM)
with ChromaMcpManager, a singleton stdio MCP client that communicates with
chroma-mcp via uvx. This eliminates native binary issues, segfaults, and
WASM embedding failures that plagued cross-platform installs.

Key changes:
- Add ChromaMcpManager: singleton MCP client with lazy connect, auto-reconnect,
  connection lock, and Zscaler SSL cert support
- Rewrite ChromaSync to use MCP tool calls instead of chromadb npm client
- Handle chroma-mcp's non-JSON responses (plain text success/error messages)
- Treat "collection already exists" as idempotent success
- Wire ChromaMcpManager into GracefulShutdown for clean subprocess teardown
- Delete ChromaServerManager (no longer needed)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review — connection guard leak, timer leak, async reset

- Clear connecting guard in finally block to prevent permanent reconnection block
- Clear timeout after successful connection to prevent timer leak
- Make reset() async to await stop() before nullifying instance
- Delete obsolete chroma-server-manager test (imports deleted class)
- Update graceful-shutdown test to use chromaMcpManager property name

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: prevent chroma-mcp spawn storm — zombie cleanup, stale onclose guard, reconnect backoff

Three bugs caused chroma-mcp processes to accumulate (92+ observed):

1. Zombie on timeout: failed connections left subprocess alive because
   only the timer was cleared, not the transport. Now catch block
   explicitly closes transport+client before rethrowing.

2. Stale onclose race: old transport's onclose handler captured `this`
   and overwrote the current connection reference after reconnect,
   orphaning the new subprocess. Now guarded with reference check.

3. No backoff: every failure triggered immediate reconnect. With
   backfill doing hundreds of MCP calls, this created rapid-fire
   spawning. Added 10s backoff on both connection failure and
   unexpected process death.

Also includes ChromaSync fixes from PR review:
- queryChroma deduplication now preserves index-aligned arrays
- SQL injection guard on backfill ID exclusion lists

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 18:32:38 -05:00

109 lines
2.9 KiB
TypeScript

/**
* DatabaseManager: Single long-lived database connection
*
* Responsibility:
* - Manage single database connection for worker lifetime
* - Provide centralized access to SessionStore and SessionSearch
* - High-level database operations
* - ChromaSync integration
*/
import { SessionStore } from '../sqlite/SessionStore.js';
import { SessionSearch } from '../sqlite/SessionSearch.js';
import { ChromaSync } from '../sync/ChromaSync.js';
import { logger } from '../../utils/logger.js';
import type { DBSession } from '../worker-types.js';
export class DatabaseManager {
private sessionStore: SessionStore | null = null;
private sessionSearch: SessionSearch | null = null;
private chromaSync: ChromaSync | null = null;
/**
* Initialize database connection (once, stays open)
*/
async initialize(): Promise<void> {
// Open database connection (ONCE)
this.sessionStore = new SessionStore();
this.sessionSearch = new SessionSearch();
// Initialize ChromaSync (lazy - connects on first search, not at startup)
this.chromaSync = new ChromaSync('claude-mem');
logger.info('DB', 'Database initialized');
}
/**
* Close database connection and cleanup all resources
*/
async close(): Promise<void> {
// Close ChromaSync first (MCP connection lifecycle managed by ChromaMcpManager)
if (this.chromaSync) {
await this.chromaSync.close();
this.chromaSync = null;
}
if (this.sessionStore) {
this.sessionStore.close();
this.sessionStore = null;
}
if (this.sessionSearch) {
this.sessionSearch.close();
this.sessionSearch = null;
}
logger.info('DB', 'Database closed');
}
/**
* Get SessionStore instance (throws if not initialized)
*/
getSessionStore(): SessionStore {
if (!this.sessionStore) {
throw new Error('Database not initialized');
}
return this.sessionStore;
}
/**
* Get SessionSearch instance (throws if not initialized)
*/
getSessionSearch(): SessionSearch {
if (!this.sessionSearch) {
throw new Error('Database not initialized');
}
return this.sessionSearch;
}
/**
* Get ChromaSync instance (throws if not initialized)
*/
getChromaSync(): ChromaSync {
if (!this.chromaSync) {
throw new Error('ChromaSync not initialized');
}
return this.chromaSync;
}
// REMOVED: cleanupOrphanedSessions - violates "EVERYTHING SHOULD SAVE ALWAYS"
// Worker restarts don't make sessions orphaned. Sessions are managed by hooks
// and exist independently of worker state.
/**
* Get session by ID (throws if not found)
*/
getSessionById(sessionDbId: number): {
id: number;
content_session_id: string;
memory_session_id: string | null;
project: string;
user_prompt: string;
} {
const session = this.getSessionStore().getSessionById(sessionDbId);
if (!session) {
throw new Error(`Session ${sessionDbId} not found`);
}
return session;
}
}