0e502dbd21
* feat: add smart-file-read module for token-optimized semantic code search - Created package.json for the smart-file-read module with dependencies and scripts. - Implemented parser.ts for code structure parsing using tree-sitter, supporting multiple languages. - Developed search.ts for searching code files and symbols with grep-style and structural matching. - Added test-run.mjs for testing search and outline functionalities. - Configured TypeScript with tsconfig.json for strict type checking and module resolution. * fix: update .gitignore to include _tree-sitter and remove unused subproject * feat: add preliminary results and skill recommendation for smart-explore module * chore: remove outdated plan.md file detailing session start hook issues * feat: update Smart File Read integration plan and skill documentation for smart-explore * feat: migrate Smart File Read to web-tree-sitter WASM for cross-platform compatibility * refactor: switch to tree-sitter CLI for parsing and enhance search functionality - Updated `parser.ts` to utilize the tree-sitter CLI for AST extraction instead of native bindings, improving compatibility and performance. - Removed grammar loading logic and replaced it with a path resolution for grammar packages. - Implemented batch parsing in `parseFilesBatch` to handle multiple files in a single CLI call, enhancing search speed. - Refactored `searchCodebase` to collect files and parse them in batches, streamlining the search process. - Adjusted symbol extraction logic to accommodate the new parsing method and ensure accurate symbol matching. * feat: update Smart File Read integration plan to utilize tree-sitter CLI for improved performance and cross-platform compatibility * feat: add smart-file-read parser and search to src/services Copy validated tree-sitter CLI-based parser and search modules from smart-file-read prototype into the claude-mem source tree for MCP tool integration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: register smart_search, smart_unfold, smart_outline MCP tools Add 3 tree-sitter AST-based code exploration tools to the MCP server. Direct execution (no HTTP delegation) — they call parser/search functions directly for sub-second response times. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add tree-sitter CLI deps to build system and plugin runtime Externalize tree-sitter packages in esbuild MCP server build. Add 10 grammar packages + CLI to plugin package.json for runtime install. Remove unused @chroma-core/default-embed from plugin deps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: create smart-explore skill with 3-layer workflow docs Progressive disclosure workflow: search -> outline -> unfold. Documents all 3 MCP tools with parameters and token economics. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add comprehensive documentation for the smart-explore feature - Introduced a detailed technical reference covering the architecture, parser, search engine, and tool registration for the smart-explore feature in claude-mem. - Documented the three-layer workflow: search, outline, and unfold, along with their respective MCP tools. - Explained the parsing process using tree-sitter, including language support, query patterns, and symbol extraction. - Outlined the search module's functionality, including file discovery, batch parsing, and relevance scoring. - Provided insights into build system integration and token economics for efficient code exploration. * chore: remove experiment artifacts, prototypes, and plan files Remove A/B test docs, prototype smart-file-read directory, and implementation plans. Keep only production code. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: simplify hooks configuration and remove setup script * fix: use execFileSync to prevent command injection in tree-sitter parser Replaces execSync shell string with execFileSync + argument array, eliminating shell interpretation of file paths. Also corrects file_pattern description from "Glob pattern" to "Substring filter". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
142 lines
5.3 KiB
Markdown
142 lines
5.3 KiB
Markdown
---
|
|
name: smart-explore
|
|
description: Token-optimized structural code search using tree-sitter AST parsing. Use instead of reading full files when you need to understand code structure, find functions, or explore a codebase efficiently.
|
|
---
|
|
|
|
# Smart Explore
|
|
|
|
Structural code exploration using AST parsing. **This skill overrides your default exploration behavior.** While this skill is active, use smart_search/smart_outline/smart_unfold as your primary tools instead of Read, Grep, and Glob.
|
|
|
|
## Your Next Tool Call
|
|
|
|
This skill only loads instructions. You must call the MCP tools yourself. Your next action should be one of:
|
|
|
|
```
|
|
smart_search(query="<topic>", path="./src") -- discover files + symbols across a directory
|
|
smart_outline(file_path="<file>") -- structural skeleton of one file
|
|
smart_unfold(file_path="<file>", symbol_name="<name>") -- full source of one symbol
|
|
```
|
|
|
|
Do NOT run Grep, Glob, Read, or find to discover files first. `smart_search` walks directories, parses all code files, and returns ranked symbols in one call. It replaces the Glob → Grep → Read discovery cycle.
|
|
|
|
## 3-Layer Workflow
|
|
|
|
### Step 1: Search -- Discover Files and Symbols
|
|
|
|
```
|
|
smart_search(query="shutdown", path="./src", max_results=15)
|
|
```
|
|
|
|
**Returns:** Ranked symbols with signatures, line numbers, match reasons, plus folded file views (~2-6k tokens)
|
|
|
|
```
|
|
-- Matching Symbols --
|
|
function performGracefulShutdown (services/infrastructure/GracefulShutdown.ts:56)
|
|
function httpShutdown (services/infrastructure/HealthMonitor.ts:92)
|
|
method WorkerService.shutdown (services/worker-service.ts:846)
|
|
|
|
-- Folded File Views --
|
|
services/infrastructure/GracefulShutdown.ts (7 symbols)
|
|
services/worker-service.ts (12 symbols)
|
|
```
|
|
|
|
This is your discovery tool. It finds relevant files AND shows their structure. No Glob/find pre-scan needed.
|
|
|
|
**Parameters:**
|
|
|
|
- `query` (string, required) -- What to search for (function name, concept, class name)
|
|
- `path` (string) -- Root directory to search (defaults to cwd)
|
|
- `max_results` (number) -- Max matching symbols, default 20, max 50
|
|
- `file_pattern` (string, optional) -- Filter to specific files/paths
|
|
|
|
### Step 2: Outline -- Get File Structure
|
|
|
|
```
|
|
smart_outline(file_path="services/worker-service.ts")
|
|
```
|
|
|
|
**Returns:** Complete structural skeleton -- all functions, classes, methods, properties, imports (~1-2k tokens per file)
|
|
|
|
**Skip this step** when Step 1's folded file views already provide enough structure. Most useful for files not covered by the search results.
|
|
|
|
**Parameters:**
|
|
|
|
- `file_path` (string, required) -- Path to the file
|
|
|
|
### Step 3: Unfold -- See Implementation
|
|
|
|
Review symbols from Steps 1-2. Pick the ones you need. Unfold only those:
|
|
|
|
```
|
|
smart_unfold(file_path="services/worker-service.ts", symbol_name="shutdown")
|
|
```
|
|
|
|
**Returns:** Full source code of the specified symbol including JSDoc, decorators, and complete implementation (~1-7k tokens depending on symbol size)
|
|
|
|
**Parameters:**
|
|
|
|
- `file_path` (string, required) -- Path to the file (as returned by search/outline)
|
|
- `symbol_name` (string, required) -- Name of the function/class/method to expand
|
|
|
|
## When to Use Standard Tools Instead
|
|
|
|
Use these only when smart_* tools are the wrong fit:
|
|
|
|
- **Grep:** Exact string/regex search ("find all TODO comments", "where is `ensureWorkerStarted` defined?")
|
|
- **Read:** Small files under ~100 lines, non-code files (JSON, markdown, config)
|
|
- **Glob:** File path patterns ("find all test files")
|
|
|
|
For code files over ~100 lines, prefer smart_outline + smart_unfold over Read.
|
|
|
|
## Workflow Examples
|
|
|
|
**Discover how a feature works (cross-cutting):**
|
|
|
|
```
|
|
1. smart_search(query="shutdown", path="./src")
|
|
-> 14 symbols across 7 files, full picture in one call
|
|
2. smart_unfold(file_path="services/infrastructure/GracefulShutdown.ts", symbol_name="performGracefulShutdown")
|
|
-> See the core implementation
|
|
```
|
|
|
|
**Navigate a large file:**
|
|
|
|
```
|
|
1. smart_outline(file_path="services/worker-service.ts")
|
|
-> 1,466 tokens: 12 functions, WorkerService class with 24 members
|
|
2. smart_unfold(file_path="services/worker-service.ts", symbol_name="startSessionProcessor")
|
|
-> 1,610 tokens: the specific method you need
|
|
Total: ~3,076 tokens vs ~12,000 to Read the full file
|
|
```
|
|
|
|
**Write documentation about code (hybrid workflow):**
|
|
|
|
```
|
|
1. smart_search(query="feature name", path="./src") -- discover all relevant files and symbols
|
|
2. smart_outline on key files -- understand structure
|
|
3. smart_unfold on important functions -- get implementation details
|
|
4. Read on small config/markdown/plan files -- get non-code context
|
|
```
|
|
|
|
Use smart_* tools for code exploration, Read for non-code files. Mix freely.
|
|
|
|
**Exploration then precision:**
|
|
|
|
```
|
|
1. smart_search(query="session", path="./src", max_results=10)
|
|
-> 10 ranked symbols: SessionMetadata, SessionQueueProcessor, SessionSummary...
|
|
2. Pick the relevant one, unfold it
|
|
```
|
|
|
|
## Token Economics
|
|
|
|
| Approach | Tokens | Use Case |
|
|
|----------|--------|----------|
|
|
| smart_outline | ~1,500 | "What's in this file?" |
|
|
| smart_unfold | ~1,600 | "Show me this function" |
|
|
| smart_search | ~2,000-6,000 | "How does X work?" |
|
|
| Read (full file) | ~12,000+ | When you truly need everything |
|
|
| Explore agent | ~20,000-40,000 | Same as smart_search, 6-12x more expensive |
|
|
|
|
**8x savings** on file understanding (outline + unfold vs Read). **6-12x savings** on exploration vs Explore agent.
|