feat: replace WASM embeddings with persistent chroma-mcp MCP connection (#1176)
* feat: replace WASM embeddings with persistent chroma-mcp MCP connection Replace ChromaServerManager (npx chroma run + chromadb npm + ONNX/WASM) with ChromaMcpManager, a singleton stdio MCP client that communicates with chroma-mcp via uvx. This eliminates native binary issues, segfaults, and WASM embedding failures that plagued cross-platform installs. Key changes: - Add ChromaMcpManager: singleton MCP client with lazy connect, auto-reconnect, connection lock, and Zscaler SSL cert support - Rewrite ChromaSync to use MCP tool calls instead of chromadb npm client - Handle chroma-mcp's non-JSON responses (plain text success/error messages) - Treat "collection already exists" as idempotent success - Wire ChromaMcpManager into GracefulShutdown for clean subprocess teardown - Delete ChromaServerManager (no longer needed) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — connection guard leak, timer leak, async reset - Clear connecting guard in finally block to prevent permanent reconnection block - Clear timeout after successful connection to prevent timer leak - Make reset() async to await stop() before nullifying instance - Delete obsolete chroma-server-manager test (imports deleted class) - Update graceful-shutdown test to use chromaMcpManager property name Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: prevent chroma-mcp spawn storm — zombie cleanup, stale onclose guard, reconnect backoff Three bugs caused chroma-mcp processes to accumulate (92+ observed): 1. Zombie on timeout: failed connections left subprocess alive because only the timer was cleared, not the transport. Now catch block explicitly closes transport+client before rethrowing. 2. Stale onclose race: old transport's onclose handler captured `this` and overwrote the current connection reference after reconnect, orphaning the new subprocess. Now guarded with reference check. 3. No backoff: every failure triggered immediate reconnect. With backfill doing hundreds of MCP calls, this created rapid-fire spawning. Added 10s backoff on both connection failure and unexpected process death. Also includes ChromaSync fixes from PR review: - queryChroma deduplication now preserves index-aligned arrays - SQL injection guard on backfill ID exclusion lists Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -87,9 +87,9 @@ describe('GracefulShutdown', () => {
|
||||
})
|
||||
};
|
||||
|
||||
const mockChromaServer = {
|
||||
const mockChromaMcpManager = {
|
||||
stop: mock(async () => {
|
||||
callOrder.push('chromaServer.stop');
|
||||
callOrder.push('chromaMcpManager.stop');
|
||||
})
|
||||
};
|
||||
|
||||
@@ -102,7 +102,7 @@ describe('GracefulShutdown', () => {
|
||||
sessionManager: mockSessionManager,
|
||||
mcpClient: mockMcpClient,
|
||||
dbManager: mockDbManager,
|
||||
chromaServer: mockChromaServer
|
||||
chromaMcpManager: mockChromaMcpManager
|
||||
};
|
||||
|
||||
await performGracefulShutdown(config);
|
||||
@@ -112,7 +112,7 @@ describe('GracefulShutdown', () => {
|
||||
expect(callOrder).toContain('serverClose');
|
||||
expect(callOrder).toContain('sessionManager.shutdownAll');
|
||||
expect(callOrder).toContain('mcpClient.close');
|
||||
expect(callOrder).toContain('chromaServer.stop');
|
||||
expect(callOrder).toContain('chromaMcpManager.stop');
|
||||
expect(callOrder).toContain('dbManager.close');
|
||||
|
||||
// Verify server closes before session manager
|
||||
@@ -125,7 +125,7 @@ describe('GracefulShutdown', () => {
|
||||
expect(callOrder.indexOf('mcpClient.close')).toBeLessThan(callOrder.indexOf('dbManager.close'));
|
||||
|
||||
// Verify Chroma stops before DB closes
|
||||
expect(callOrder.indexOf('chromaServer.stop')).toBeLessThan(callOrder.indexOf('dbManager.close'));
|
||||
expect(callOrder.indexOf('chromaMcpManager.stop')).toBeLessThan(callOrder.indexOf('dbManager.close'));
|
||||
});
|
||||
|
||||
it('should remove PID file during shutdown', async () => {
|
||||
@@ -216,9 +216,9 @@ describe('GracefulShutdown', () => {
|
||||
})
|
||||
};
|
||||
|
||||
const mockChromaServer = {
|
||||
const mockChromaMcpManager = {
|
||||
stop: mock(async () => {
|
||||
callOrder.push('chromaServer');
|
||||
callOrder.push('chromaMcpManager');
|
||||
})
|
||||
};
|
||||
|
||||
@@ -227,12 +227,12 @@ describe('GracefulShutdown', () => {
|
||||
sessionManager: mockSessionManager,
|
||||
mcpClient: mockMcpClient,
|
||||
dbManager: mockDbManager,
|
||||
chromaServer: mockChromaServer
|
||||
chromaMcpManager: mockChromaMcpManager
|
||||
};
|
||||
|
||||
await performGracefulShutdown(config);
|
||||
|
||||
expect(callOrder).toEqual(['sessionManager', 'mcpClient', 'chromaServer', 'dbManager']);
|
||||
expect(callOrder).toEqual(['sessionManager', 'mcpClient', 'chromaMcpManager', 'dbManager']);
|
||||
});
|
||||
|
||||
it('should handle shutdown when PID file does not exist', async () => {
|
||||
|
||||
Reference in New Issue
Block a user