Files
claude-mem/scripts/build-hooks.js
T
Alex Newman c5e68a17c8 refactor: Clean up search architecture, remove experimental contextualize endpoint (#133)
* Refactor code structure for improved readability and maintainability

* Add test results for search API and related functionalities

- Created test result files for various search-related functionalities, including:
  - test-11-search-server-changes.json
  - test-12-context-hook-changes.json
  - test-13-worker-service-changes.json
  - test-14-patterns.json
  - test-15-gotchas.json
  - test-16-discoveries.json
  - test-17-all-bugfixes.json
  - test-18-all-features.json
  - test-19-all-decisions.json
  - test-20-session-search.json
  - test-21-prompt-search.json
  - test-22-decisions-endpoint.json
  - test-23-changes-endpoint.json
  - test-24-how-it-works-endpoint.json
  - test-25-contextualize-endpoint.json
  - test-26-timeline-around-observation.json
  - test-27-multi-param-combo.json
  - test-28-file-type-combo.json

- Each test result file captures specific search failures or outcomes, including issues with undefined properties and successful execution of search queries.
- Enhanced documentation of search architecture and testing strategies, ensuring compliance with established guidelines and improving overall search functionality.

* feat: Enhance unified search API with catch-all parameters and backward compatibility

- Implemented a unified search API at /api/search that accepts catch-all parameters for filtering by type, observation type, concepts, and files.
- Maintained backward compatibility by keeping granular endpoints functional while routing through the same infrastructure.
- Completed comprehensive testing of search capabilities with real-world query scenarios.

fix: Address missing debug output in search API query tests

- Flushed PM2 logs and executed search queries to verify functionality.
- Diagnosed absence of "Raw Chroma" debug messages in worker logs, indicating potential issues with logging or query processing.

refactor: Improve build and deployment pipeline for claude-mem plugin

- Successfully built and synced all hooks and services to the marketplace directory.
- Ensured all dependencies are installed and up-to-date in the deployment location.

feat: Implement hybrid search filters with 90-day recency window

- Enhanced search server to apply a 90-day recency filter to Chroma results before categorizing by document type.

fix: Correct parameter handling in searchUserPrompts method

- Added support for filter-only queries and improved dual-path logic for clarity.

refactor: Rename FTS5 method to clarify fallback status

- Renamed escapeFTS5 to escapeFTS5_fallback_when_chroma_unavailable to indicate its temporary usage.

feat: Introduce contextualize tool for comprehensive project overview

- Added a new tool to fetch recent observations, sessions, and user prompts, providing a quick project overview.

feat: Add semantic shortcut tools for common search patterns

- Implemented 'decisions', 'changes', and 'how_it_works' tools for convenient access to frequently searched observation categories.

feat: Unified timeline tool supports anchor and query modes

- Combined get_context_timeline and get_timeline_by_query into a single interface for timeline exploration.

feat: Unified search tool added to MCP server

- New tool queries all memory types simultaneously, providing combined chronological results for improved search efficiency.

* Refactor search functionality to clarify FTS5 fallback usage

- Updated `worker-service.cjs` to replace FTS5 fallback function with a more descriptive name and improved error handling.
- Enhanced documentation in `SKILL.md` to specify the unified API endpoint and clarify the behavior of the search engine, including the conditions under which FTS5 is used.
- Modified `search-server.ts` to provide clearer logging and descriptions regarding the fallback to FTS5 when UVX/Python is unavailable.
- Renamed and updated the `SessionSearch.ts` methods to reflect the conditions for using FTS5, emphasizing the lack of semantic understanding in fallback scenarios.

* feat: Add ID-based fetch endpoints and simplify mem-search skill

**Problem:**
- Search returns IDs but no way to fetch by ID
- Skill documentation was bloated with too many options
- Claude wasn't using IDs because we didn't tell it how

**Solution:**
1. Added three new HTTP endpoints:
   - GET /api/observation/:id
   - GET /api/session/:id
   - GET /api/prompt/:id

2. Completely rewrote SKILL.md:
   - Stripped complexity down to essentials
   - Clear 3-step prescriptive workflow: Search → Review IDs → Fetch by ID
   - Emphasized ID usage: "The IDs are there for a reason - USE THEM"
   - Removed confusing multi-endpoint documentation
   - Kept only unified search with filters

**Impact:**
- Token efficiency: Claude can now fetch full details only for relevant IDs
- Clarity: One clear workflow instead of 10+ options to choose from
- Usability: IDs are no longer wasted context - they're actionable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: Move internal docs to private directory

Moved POSTMORTEM and planning docs to ./private to exclude from PR reviews.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove experimental contextualize endpoint

- Removed contextualize MCP tool from search-server (saves ~4KB)
- Disabled FTS5 fallback paths in SessionSearch (now vector-first)
- Cleaned up CLAUDE.md documentation
- Removed contextualize-rewrite-plan.md doc

Rationale:
- Contextualize is better suited as a skill (LLM-powered) than an endpoint
- Search API already provides vector search with configurable limits
- Created issue #132 to track future contextualize skill implementation

Changes:
- src/servers/search-server.ts: Removed contextualize tool definition
- src/services/sqlite/SessionSearch.ts: Disabled FTS5 fallback, added deprecation warnings
- CLAUDE.md: Cleaned up outdated skill documentation
- docs/: Removed contextualize plan document

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Complete FTS5 cleanup - remove all deprecated search code

This completes the FTS5 cleanup work by removing all commented-out
FTS5 search code while preserving database tables for backward compatibility.

Changes:
- Removed 200+ lines of commented FTS5 search code from SessionSearch.ts
- Removed deprecated degraded_search_query__when_uvx_unavailable method
- Updated all method documentation to clarify vector-first architecture
- Updated class documentation to reflect filter-only query support
- Updated CLAUDE.md to remove FTS5 search references
- Clarified that FTS5 tables exist for backward compatibility only
- Updated "Why SQLite FTS5" section to "Why Vector-First Search"

Database impact: NONE - FTS5 tables remain intact for existing installations

Search architecture:
- ChromaDB: All text-based vector search queries
- SQLite: Filter-only queries (date ranges, metadata, no query text)
- FTS5 tables: Maintained but unused (backward compatibility)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove all FTS5 fallback execution code from search-server

Completes the FTS5 cleanup by removing all fallback execution paths
that attempted to use FTS5 when ChromaDB was unavailable.

Changes:
- Removed all FTS5 fallback code execution paths
- When ChromaDB fails or is unavailable, return empty results with helpful error messages
- Updated all deprecated tool descriptions (search_observations, search_sessions, search_user_prompts)
- Changed error messages to indicate FTS5 fallback has been removed
- Added installation instructions for UVX/Python when vector search is unavailable
- Updated comments from "hybrid search" to "vector-first search"
- Removed ~100 lines of dead FTS5 fallback code

Database impact: NONE - FTS5 tables remain intact (backward compatibility)

Search behavior when ChromaDB unavailable:
- Text queries: Return empty results with error explaining ChromaDB is required
- Filter-only queries (no text): Continue to work via direct SQLite

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Address PR 133 review feedback

Critical fixes:
- Remove contextualize endpoint from worker-service (route + handler)
- Fix build script logging to show correct .cjs extension (was .mjs)

Documentation improvements:
- Add comprehensive FTS5 retention rationale documentation
- Include v7.0.0 removal TODO for future cleanup

Testing:
- Build succeeds with correct output logging
- Worker restarts successfully (30th restart)
- Contextualize endpoint properly removed (404 response)
- Search endpoint verified working

This addresses all critical review feedback from PR 133.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-21 18:59:23 -05:00

171 lines
5.1 KiB
JavaScript

#!/usr/bin/env node
/**
* Build script for claude-mem hooks
* Bundles TypeScript hooks into individual standalone executables using esbuild
*/
import { build } from 'esbuild';
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const HOOKS = [
{ name: 'context-hook', source: 'src/hooks/context-hook.ts' },
{ name: 'new-hook', source: 'src/hooks/new-hook.ts' },
{ name: 'save-hook', source: 'src/hooks/save-hook.ts' },
{ name: 'summary-hook', source: 'src/hooks/summary-hook.ts' },
{ name: 'cleanup-hook', source: 'src/hooks/cleanup-hook.ts' },
{ name: 'user-message-hook', source: 'src/hooks/user-message-hook.ts' }
];
const WORKER_SERVICE = {
name: 'worker-service',
source: 'src/services/worker-service.ts'
};
const SEARCH_SERVER = {
name: 'search-server',
source: 'src/servers/search-server.ts'
};
async function buildHooks() {
console.log('🔨 Building claude-mem hooks and worker service...\n');
try {
// Read version from package.json
const packageJson = JSON.parse(fs.readFileSync('package.json', 'utf-8'));
const version = packageJson.version;
console.log(`📌 Version: ${version}`);
// Create output directories
console.log('\n📦 Preparing output directories...');
const hooksDir = 'plugin/scripts';
const uiDir = 'plugin/ui';
if (!fs.existsSync(hooksDir)) {
fs.mkdirSync(hooksDir, { recursive: true });
}
if (!fs.existsSync(uiDir)) {
fs.mkdirSync(uiDir, { recursive: true });
}
console.log('✓ Output directories ready');
// Build React viewer
console.log('\n📋 Building React viewer...');
const { spawn } = await import('child_process');
const viewerBuild = spawn('node', ['scripts/build-viewer.js'], { stdio: 'inherit' });
await new Promise((resolve, reject) => {
viewerBuild.on('exit', (code) => {
if (code === 0) {
resolve();
} else {
reject(new Error(`Viewer build failed with exit code ${code}`));
}
});
});
// Build worker service
console.log(`\n🔧 Building worker service...`);
await build({
entryPoints: [WORKER_SERVICE.source],
bundle: true,
platform: 'node',
target: 'node18',
format: 'cjs',
outfile: `${hooksDir}/${WORKER_SERVICE.name}.cjs`,
minify: true,
logLevel: 'error', // Suppress warnings (import.meta warning is benign)
external: ['better-sqlite3'],
define: {
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
},
banner: {
js: '#!/usr/bin/env node'
}
});
// Make worker service executable
fs.chmodSync(`${hooksDir}/${WORKER_SERVICE.name}.cjs`, 0o755);
const workerStats = fs.statSync(`${hooksDir}/${WORKER_SERVICE.name}.cjs`);
console.log(`✓ worker-service built (${(workerStats.size / 1024).toFixed(2)} KB)`);
// Build search server
console.log(`\n🔧 Building search server...`);
await build({
entryPoints: [SEARCH_SERVER.source],
bundle: true,
platform: 'node',
target: 'node18',
format: 'cjs',
outfile: `${hooksDir}/${SEARCH_SERVER.name}.cjs`,
minify: true,
logLevel: 'error',
external: ['better-sqlite3'],
define: {
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
},
banner: {
js: '#!/usr/bin/env node'
}
});
// Make search server executable
fs.chmodSync(`${hooksDir}/${SEARCH_SERVER.name}.cjs`, 0o755);
const searchServerStats = fs.statSync(`${hooksDir}/${SEARCH_SERVER.name}.cjs`);
console.log(`✓ search-server built (${(searchServerStats.size / 1024).toFixed(2)} KB)`);
// Build each hook
for (const hook of HOOKS) {
console.log(`\n🔧 Building ${hook.name}...`);
const outfile = `${hooksDir}/${hook.name}.js`;
await build({
entryPoints: [hook.source],
bundle: true,
platform: 'node',
target: 'node18',
format: 'esm',
outfile,
minify: true,
external: ['better-sqlite3'],
define: {
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
},
banner: {
js: '#!/usr/bin/env node'
}
});
// Make executable
fs.chmodSync(outfile, 0o755);
// Check file size
const stats = fs.statSync(outfile);
const sizeInKB = (stats.size / 1024).toFixed(2);
console.log(`${hook.name} built (${sizeInKB} KB)`);
}
console.log('\n✅ All hooks, worker service, and search server built successfully!');
console.log(` Output: ${hooksDir}/`);
console.log(` - Hooks: *-hook.js`);
console.log(` - Worker: worker-service.cjs`);
console.log(` - Search Server: search-server.cjs`);
console.log(` - Skills: plugin/skills/`);
console.log('\n💡 Note: Dependencies will be auto-installed on first hook execution');
} catch (error) {
console.error('\n❌ Build failed:', error.message);
if (error.errors) {
console.error('\nBuild errors:');
error.errors.forEach(err => console.error(` - ${err.text}`));
}
process.exit(1);
}
}
buildHooks();