feat: expand smart-explore to 24 languages with markdown support and user-installable grammars

Add 15 new tree-sitter language grammars (Kotlin, Swift, PHP, Elixir, Lua, Scala,
Bash, Haskell, Zig, CSS, SCSS, TOML, YAML, SQL, Markdown) with verified SCM queries.
Add markdown-specific formatting with heading hierarchy, code block detection, and
section-aware unfold. Add user-installable grammar system via .claude-mem.json config
with custom query file support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alex Newman
2026-04-07 13:24:56 -07:00
parent 9f01228a2b
commit 95889c7b4e
7 changed files with 641 additions and 35 deletions
+48
View File
@@ -125,3 +125,51 @@ get_observations(ids=[11131, 10942, 10855], orderBy="date_desc")
- **Full observation:** ~500-1000 tokens each
- **Batch fetch:** 1 HTTP request vs N individual requests
- **10x token savings** by filtering before fetching
## Smart-Explore Language Support
Smart-explore tools (`smart_search`, `smart_outline`, `smart_unfold`) use tree-sitter AST parsing. The following languages are supported out of the box.
### 24 Bundled Languages
JS, TS, Python, Go, Rust, Ruby, Java, C, C++, Kotlin, Swift, PHP, Elixir, Lua, Scala, Bash, Haskell, Zig, CSS, SCSS, TOML, YAML, SQL, Markdown
### Markdown Special Support
Markdown files get structure-aware parsing beyond generic tree-sitter:
- **Heading hierarchy** -- `#`/`##`/`###` headings are extracted as nested symbols (sections contain subsections)
- **Code block detection** -- fenced code blocks are surfaced as `code` symbols with language annotation
- **Section-aware unfold** -- `smart_unfold` on a heading returns the full section content (heading through all subsections until the next heading of equal or higher level)
### User-Installable Grammars via `.claude-mem.json`
Add custom tree-sitter grammars for languages not in the bundled set. Place `.claude-mem.json` in the project root:
```json
{
"grammars": {
"gleam": {
"package": "tree-sitter-gleam",
"extensions": [".gleam"]
},
"protobuf": {
"package": "tree-sitter-proto",
"extensions": [".proto"],
"query": ".claude-mem/queries/proto.scm"
}
}
}
```
**Fields:**
- `package` (string, required) -- npm package name for the tree-sitter grammar
- `extensions` (array of strings, required) -- file extensions to associate with this language
- `query` (string, optional) -- path to a custom `.scm` query file for symbol extraction. If omitted, a generic query is used.
**Rules:**
- User grammars do NOT override bundled languages. If a language is already bundled, the entry is ignored.
- The npm package must be installed in the project (`npm install tree-sitter-gleam`).
- Config is cached per project root. Changes to `.claude-mem.json` take effect on next worker restart.