fix: align IDs with metadatas in ChromaSearchStrategy
ChromaSync.queryChroma() returns deduplicated sqlite_ids but the metadatas array contains multiple entries per observation (narrative + facts). The filterByRecency() method was iterating over metadatas and using the index to access ids, causing array out-of-bounds access. The fix builds a Map from sqlite_id to metadata, then iterates over the deduplicated ids array to ensure proper alignment. Symptoms before fix: - Semantic search returning incorrect/empty results - Search only working with near-exact queries - Recent items (same day) not being found Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
committed by
Alex Newman
parent
f62b5d248c
commit
9bd56c993c
@@ -167,6 +167,13 @@ export class ChromaSearchStrategy extends BaseSearchStrategy implements SearchSt
|
||||
|
||||
/**
|
||||
* Filter results by recency (90-day window)
|
||||
*
|
||||
* IMPORTANT: ChromaSync.queryChroma() returns deduplicated `ids` (unique sqlite_ids)
|
||||
* but the `metadatas` array may contain multiple entries per sqlite_id (e.g., one
|
||||
* observation can have narrative + multiple facts as separate Chroma documents).
|
||||
*
|
||||
* This method iterates over the deduplicated `ids` and finds the first matching
|
||||
* metadata for each ID to avoid array misalignment issues.
|
||||
*/
|
||||
private filterByRecency(chromaResults: {
|
||||
ids: number[];
|
||||
@@ -174,10 +181,19 @@ export class ChromaSearchStrategy extends BaseSearchStrategy implements SearchSt
|
||||
}): Array<{ id: number; meta: ChromaMetadata }> {
|
||||
const cutoff = Date.now() - SEARCH_CONSTANTS.RECENCY_WINDOW_MS;
|
||||
|
||||
return chromaResults.metadatas
|
||||
.map((meta, idx) => ({
|
||||
id: chromaResults.ids[idx],
|
||||
meta
|
||||
// Build a map from sqlite_id to first metadata for efficient lookup
|
||||
const metadataByIdMap = new Map<number, ChromaMetadata>();
|
||||
for (const meta of chromaResults.metadatas) {
|
||||
if (meta?.sqlite_id !== undefined && !metadataByIdMap.has(meta.sqlite_id)) {
|
||||
metadataByIdMap.set(meta.sqlite_id, meta);
|
||||
}
|
||||
}
|
||||
|
||||
// Iterate over deduplicated ids and get corresponding metadata
|
||||
return chromaResults.ids
|
||||
.map(id => ({
|
||||
id,
|
||||
meta: metadataByIdMap.get(id) as ChromaMetadata
|
||||
}))
|
||||
.filter(item => item.meta && item.meta.created_at_epoch > cutoff);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user