Mem-search enhancements: table output, simplified API, Sonnet default, and removed fake URIs (#317)
* feat: Add batch fetching for observations and update documentation - Implemented a new endpoint for fetching multiple observations by IDs in a single request. - Updated the DataRoutes to include a POST /api/observations/batch endpoint. - Enhanced SKILL.md documentation to reflect changes in the search process and batch fetching capabilities. - Increased the default limit for search results from 5 to 40 for better usability. * feat!: Fix timeline parameter passing with SearchManager alignment BREAKING CHANGE: Timeline MCP tools now use standardized parameter names - anchor_id → anchor - before → depth_before - after → depth_after - obs_type → type (timeline tool only) Fixes timeline endpoint failures caused by parameter name mismatch between MCP layer and SearchManager. Adds new SessionStore methods for fetching prompts and session summaries by ID. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * docs: reframe timeline parameter fix as bug fix, not breaking change The timeline tools were completely broken due to parameter name mismatch. There's nothing to migrate from since the old parameters never worked. Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com> * Refactor mem-search documentation and optimize API tool definitions - Updated SKILL.md to emphasize batch fetching for observations, clarifying usage and efficiency. - Removed deprecated tools from mcp-server.ts and streamlined tool definitions for clarity. - Enhanced formatting in FormattingService.ts for better output readability. - Adjusted SearchManager.ts to improve result headers and removed unnecessary search tips from combined text. * Refactor FormattingService and SearchManager for table-based output - Updated FormattingService to format search results as tables, including methods for formatting observations, sessions, and user prompts. - Removed JSON format handling from SearchManager and streamlined result formatting to consistently use table format. - Enhanced readability and consistency in search tips and formatting logic. - Introduced token estimation for observations and improved time formatting. * refactor: update documentation and API references for version bump and search functionalities * Refactor code structure for improved readability and maintainability * chore: change default model from haiku to sonnet 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: unify timeline formatting across search and context services Extract shared timeline formatting utilities into reusable module to align MCP search output format with context-generator's date/file-grouped format. Changes: - Create src/shared/timeline-formatting.ts with reusable utilities (parseJsonArray, formatDateTime, formatTime, formatDate, toRelativePath, extractFirstFile, groupByDate) - Refactor context-generator.ts to use shared utilities - Update SearchManager.search() to use date/file grouping - Add search-specific row formatters to FormattingService - Fix timeline methods to extract actual file paths from metadata instead of hardcoding 'General' - Remove Work column from search output (kept in context output) Result: Consistent date/file-grouped markdown formatting across both systems while maintaining their different column requirements. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: remove redundant legend from search output Remove legend from search/timeline results since it's already shown in SessionStart context. Saves ~30 tokens per search result. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * Refactor session summary rendering to remove links - Removed link generation for session summaries in context generation and search manager. - Updated output formatting to exclude links while maintaining the session summary structure. - Adjusted related components in TimelineService to ensure consistency across the application. * fix: move skillPath declaration outside try block to fix scoping bug The skillPath variable was declared inside the try block but referenced in the catch block for error logging. Since const is block-scoped, this would cause a ReferenceError when the error handler executes. Moved skillPath declaration before the try block so it's accessible in both try and catch scopes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix: address PR #317 code review feedback **Critical Fixes:** - Replace happy_path_error__with_fallback debug calls with proper logger methods in mcp-server.ts - All HTTP API calls now use logger.debug/error for consistent logging **Code Quality Improvements:** - Extract 90-day recency window magic numbers to named constants - Added RECENCY_WINDOW_DAYS and RECENCY_WINDOW_MS constants in SearchManager **Documentation:** - Document model cost implications of Haiku → Sonnet upgrade in CHANGELOG - Provide clear migration path for users who want to revert to Haiku 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: simplify CHANGELOG - remove cost documentation Removed model cost comparison documentation per user feedback. Kept only the technical code quality improvements. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
This commit is contained in:
Alex Newman
@@ -1,12 +1,13 @@
|
||||
/**
|
||||
* FormattingService - Handles all formatting logic for search results
|
||||
* Extracted from mcp-server.ts to follow worker service organization pattern
|
||||
* Uses table format matching context-generator style for visual consistency
|
||||
*/
|
||||
|
||||
import { ObservationSearchResult, SessionSummarySearchResult, UserPromptSearchResult } from '../sqlite/types.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { TYPE_ICON_MAP, TYPE_WORK_EMOJI_MAP } from '../../constants/observation-metadata.js';
|
||||
|
||||
export type FormatType = 'index' | 'full';
|
||||
// Token estimation constant (matches context-generator)
|
||||
const CHARS_PER_TOKEN_ESTIMATE = 4;
|
||||
|
||||
export class FormattingService {
|
||||
/**
|
||||
@@ -15,232 +16,155 @@ export class FormattingService {
|
||||
formatSearchTips(): string {
|
||||
return `\n---
|
||||
💡 Search Strategy:
|
||||
ALWAYS search with index format FIRST to get an overview and identify relevant results.
|
||||
This is critical for token efficiency - index format uses ~10x fewer tokens than full format.
|
||||
1. Search with index to see titles, dates, IDs
|
||||
2. Use timeline to get context around interesting results
|
||||
3. Batch fetch full details: get_batch_observations(ids=[...])
|
||||
|
||||
Search workflow:
|
||||
1. Initial search: Use default (index) format to see titles, dates, and sources
|
||||
2. Review results: Identify which items are most relevant to your needs
|
||||
3. Deep dive: Only then use format: "full" on specific items of interest
|
||||
4. Narrow down: Use filters (type, dateStart/dateEnd, concepts, files) to refine results
|
||||
|
||||
Other tips:
|
||||
• To search by concept: Use find_by_concept tool
|
||||
• To browse by type: Use find_by_type with ["decision", "feature", etc.]
|
||||
• To sort by date: Use orderBy: "date_desc" or "date_asc"`;
|
||||
Tips:
|
||||
• Filter by type: obs_type="bugfix,feature"
|
||||
• Filter by date: dateStart="2025-01-01"
|
||||
• Sort: orderBy="date_desc" or "date_asc"`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format observation as index entry (title, date, ID only)
|
||||
* Format time from epoch (matches context-generator formatTime)
|
||||
*/
|
||||
formatObservationIndex(obs: ObservationSearchResult, index: number): string {
|
||||
const title = obs.title || `Observation #${obs.id}`;
|
||||
const date = new Date(obs.created_at_epoch).toLocaleString();
|
||||
const type = obs.type ? `[${obs.type}]` : '';
|
||||
|
||||
return `${index + 1}. ${type} ${title}
|
||||
Date: ${date}
|
||||
Source: claude-mem://observation/${obs.id}`;
|
||||
private formatTime(epoch: number): string {
|
||||
return new Date(epoch).toLocaleString('en-US', {
|
||||
hour: 'numeric',
|
||||
minute: '2-digit',
|
||||
hour12: true
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Format session summary as index entry (title, date, ID only)
|
||||
* Estimate read tokens for an observation
|
||||
*/
|
||||
formatSessionIndex(session: SessionSummarySearchResult, index: number): string {
|
||||
const title = session.request || `Session ${session.sdk_session_id?.substring(0, 8) || 'unknown'}`;
|
||||
const date = new Date(session.created_at_epoch).toLocaleString();
|
||||
|
||||
return `${index + 1}. ${title}
|
||||
Date: ${date}
|
||||
Source: claude-mem://session/${session.sdk_session_id}`;
|
||||
private estimateReadTokens(obs: ObservationSearchResult): number {
|
||||
const size = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
(obs.facts?.length || 0);
|
||||
return Math.ceil(size / CHARS_PER_TOKEN_ESTIMATE);
|
||||
}
|
||||
|
||||
/**
|
||||
* Format user prompt as index entry (full text - don't truncate context!)
|
||||
* Format observation as table row
|
||||
* | ID | Time | T | Title | Read | Work |
|
||||
*/
|
||||
formatUserPromptIndex(prompt: UserPromptSearchResult, index: number): string {
|
||||
const date = new Date(prompt.created_at_epoch).toLocaleString();
|
||||
formatObservationIndex(obs: ObservationSearchResult, _index: number): string {
|
||||
const id = `#${obs.id}`;
|
||||
const time = this.formatTime(obs.created_at_epoch);
|
||||
const icon = TYPE_ICON_MAP[obs.type as keyof typeof TYPE_ICON_MAP] || '•';
|
||||
const title = obs.title || 'Untitled';
|
||||
const readTokens = this.estimateReadTokens(obs);
|
||||
const workEmoji = TYPE_WORK_EMOJI_MAP[obs.type as keyof typeof TYPE_WORK_EMOJI_MAP] || '🔍';
|
||||
const workTokens = obs.discovery_tokens || 0;
|
||||
const workDisplay = workTokens > 0 ? `${workEmoji} ${workTokens}` : '-';
|
||||
|
||||
return `${index + 1}. "${prompt.prompt_text}"
|
||||
Date: ${date} | Prompt #${prompt.prompt_number}
|
||||
Source: claude-mem://user-prompt/${prompt.id}`;
|
||||
return `| ${id} | ${time} | ${icon} | ${title} | ~${readTokens} | ${workDisplay} |`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format observation as text content with metadata
|
||||
* Format session summary as table row
|
||||
* | ID | Time | T | Title | - | - |
|
||||
*/
|
||||
formatObservationResult(obs: ObservationSearchResult): string {
|
||||
const title = obs.title || `Observation #${obs.id}`;
|
||||
|
||||
// Build content from available fields
|
||||
const contentParts: string[] = [];
|
||||
contentParts.push(`## ${title}`);
|
||||
contentParts.push(`*Source: claude-mem://observation/${obs.id}*`);
|
||||
contentParts.push('');
|
||||
|
||||
if (obs.subtitle) {
|
||||
contentParts.push(`**${obs.subtitle}**`);
|
||||
contentParts.push('');
|
||||
}
|
||||
|
||||
if (obs.narrative) {
|
||||
contentParts.push(obs.narrative);
|
||||
contentParts.push('');
|
||||
}
|
||||
|
||||
if (obs.text) {
|
||||
contentParts.push(obs.text);
|
||||
contentParts.push('');
|
||||
}
|
||||
|
||||
// Add metadata
|
||||
const metadata: string[] = [];
|
||||
metadata.push(`Type: ${obs.type}`);
|
||||
|
||||
if (obs.facts) {
|
||||
try {
|
||||
const facts = JSON.parse(obs.facts);
|
||||
if (facts.length > 0) {
|
||||
metadata.push(`Facts: ${facts.join('; ')}`);
|
||||
}
|
||||
} catch (e) {
|
||||
logger.warn('FORMAT', 'Invalid JSON in facts field', { obsId: obs.id });
|
||||
}
|
||||
}
|
||||
|
||||
if (obs.concepts) {
|
||||
try {
|
||||
const concepts = JSON.parse(obs.concepts);
|
||||
if (concepts.length > 0) {
|
||||
metadata.push(`Concepts: ${concepts.join(', ')}`);
|
||||
}
|
||||
} catch (e) {
|
||||
logger.warn('FORMAT', 'Invalid JSON in concepts field', { obsId: obs.id });
|
||||
}
|
||||
}
|
||||
|
||||
if (obs.files_read || obs.files_modified) {
|
||||
const files: string[] = [];
|
||||
if (obs.files_read) {
|
||||
try {
|
||||
files.push(...JSON.parse(obs.files_read));
|
||||
} catch (e) {
|
||||
logger.warn('FORMAT', 'Invalid JSON in files_read field', { obsId: obs.id });
|
||||
}
|
||||
}
|
||||
if (obs.files_modified) {
|
||||
try {
|
||||
files.push(...JSON.parse(obs.files_modified));
|
||||
} catch (e) {
|
||||
logger.warn('FORMAT', 'Invalid JSON in files_modified field', { obsId: obs.id });
|
||||
}
|
||||
}
|
||||
if (files.length > 0) {
|
||||
metadata.push(`Files: ${[...new Set(files)].join(', ')}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (metadata.length > 0) {
|
||||
contentParts.push('---');
|
||||
contentParts.push(metadata.join(' | '));
|
||||
}
|
||||
|
||||
// Add date
|
||||
const date = new Date(obs.created_at_epoch).toLocaleString();
|
||||
contentParts.push('');
|
||||
contentParts.push(`---`);
|
||||
contentParts.push(`Date: ${date}`);
|
||||
|
||||
return contentParts.join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Format session summary as text content with metadata
|
||||
*/
|
||||
formatSessionResult(session: SessionSummarySearchResult): string {
|
||||
formatSessionIndex(session: SessionSummarySearchResult, _index: number): string {
|
||||
const id = `#S${session.id}`;
|
||||
const time = this.formatTime(session.created_at_epoch);
|
||||
const icon = '🎯';
|
||||
const title = session.request || `Session ${session.sdk_session_id?.substring(0, 8) || 'unknown'}`;
|
||||
|
||||
// Build content from available fields
|
||||
const contentParts: string[] = [];
|
||||
contentParts.push(`## ${title}`);
|
||||
contentParts.push(`*Source: claude-mem://session/${session.sdk_session_id}*`);
|
||||
contentParts.push('');
|
||||
|
||||
if (session.completed) {
|
||||
contentParts.push(`**Completed:** ${session.completed}`);
|
||||
contentParts.push('');
|
||||
}
|
||||
|
||||
if (session.learned) {
|
||||
contentParts.push(`**Learned:** ${session.learned}`);
|
||||
contentParts.push('');
|
||||
}
|
||||
|
||||
if (session.investigated) {
|
||||
contentParts.push(`**Investigated:** ${session.investigated}`);
|
||||
contentParts.push('');
|
||||
}
|
||||
|
||||
if (session.next_steps) {
|
||||
contentParts.push(`**Next Steps:** ${session.next_steps}`);
|
||||
contentParts.push('');
|
||||
}
|
||||
|
||||
if (session.notes) {
|
||||
contentParts.push(`**Notes:** ${session.notes}`);
|
||||
contentParts.push('');
|
||||
}
|
||||
|
||||
// Add metadata
|
||||
const metadata: string[] = [];
|
||||
|
||||
if (session.files_read || session.files_edited) {
|
||||
const files: string[] = [];
|
||||
if (session.files_read) {
|
||||
try {
|
||||
files.push(...JSON.parse(session.files_read));
|
||||
} catch (e) {
|
||||
logger.warn('FORMAT', 'Invalid JSON in session files_read field', { sessionId: session.sdk_session_id });
|
||||
}
|
||||
}
|
||||
if (session.files_edited) {
|
||||
try {
|
||||
files.push(...JSON.parse(session.files_edited));
|
||||
} catch (e) {
|
||||
logger.warn('FORMAT', 'Invalid JSON in session files_edited field', { sessionId: session.sdk_session_id });
|
||||
}
|
||||
}
|
||||
if (files.length > 0) {
|
||||
metadata.push(`Files: ${[...new Set(files)].join(', ')}`);
|
||||
}
|
||||
}
|
||||
|
||||
const date = new Date(session.created_at_epoch).toLocaleDateString();
|
||||
metadata.push(`Date: ${date}`);
|
||||
|
||||
if (metadata.length > 0) {
|
||||
contentParts.push('---');
|
||||
contentParts.push(metadata.join(' | '));
|
||||
}
|
||||
|
||||
return contentParts.join('\n');
|
||||
return `| ${id} | ${time} | ${icon} | ${title} | - | - |`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format user prompt as text content with metadata
|
||||
* Format user prompt as table row
|
||||
* | ID | Time | T | Title | - | - |
|
||||
*/
|
||||
formatUserPromptResult(prompt: UserPromptSearchResult): string {
|
||||
const contentParts: string[] = [];
|
||||
contentParts.push(`## User Prompt #${prompt.prompt_number}`);
|
||||
contentParts.push(`*Source: claude-mem://user-prompt/${prompt.id}*`);
|
||||
contentParts.push('');
|
||||
contentParts.push(prompt.prompt_text);
|
||||
contentParts.push('');
|
||||
contentParts.push('---');
|
||||
formatUserPromptIndex(prompt: UserPromptSearchResult, _index: number): string {
|
||||
const id = `#P${prompt.id}`;
|
||||
const time = this.formatTime(prompt.created_at_epoch);
|
||||
const icon = '💬';
|
||||
// Truncate long prompts for table display
|
||||
const title = prompt.prompt_text.length > 60
|
||||
? prompt.prompt_text.substring(0, 57) + '...'
|
||||
: prompt.prompt_text;
|
||||
|
||||
const date = new Date(prompt.created_at_epoch).toLocaleString();
|
||||
contentParts.push(`Date: ${date}`);
|
||||
return `| ${id} | ${time} | ${icon} | ${title} | - | - |`;
|
||||
}
|
||||
|
||||
return contentParts.join('\n');
|
||||
/**
|
||||
* Generate table header for observations
|
||||
*/
|
||||
formatTableHeader(): string {
|
||||
return `| ID | Time | T | Title | Read | Work |
|
||||
|-----|------|---|-------|------|------|`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate table header for search results (no Work column)
|
||||
*/
|
||||
formatSearchTableHeader(): string {
|
||||
return `| ID | Time | T | Title | Read |
|
||||
|----|------|---|-------|------|`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format observation as table row for search results (no Work column)
|
||||
*/
|
||||
formatObservationSearchRow(obs: ObservationSearchResult, lastTime: string): { row: string; time: string } {
|
||||
const id = `#${obs.id}`;
|
||||
const time = this.formatTime(obs.created_at_epoch);
|
||||
const icon = TYPE_ICON_MAP[obs.type as keyof typeof TYPE_ICON_MAP] || '•';
|
||||
const title = obs.title || 'Untitled';
|
||||
const readTokens = this.estimateReadTokens(obs);
|
||||
|
||||
// Use ditto mark if same time as previous row
|
||||
const timeDisplay = time === lastTime ? '″' : time;
|
||||
|
||||
return {
|
||||
row: `| ${id} | ${timeDisplay} | ${icon} | ${title} | ~${readTokens} |`,
|
||||
time
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Format session summary as table row for search results (no Work column)
|
||||
*/
|
||||
formatSessionSearchRow(session: SessionSummarySearchResult, lastTime: string): { row: string; time: string } {
|
||||
const id = `#S${session.id}`;
|
||||
const time = this.formatTime(session.created_at_epoch);
|
||||
const icon = '🎯';
|
||||
const title = session.request || `Session ${session.sdk_session_id?.substring(0, 8) || 'unknown'}`;
|
||||
|
||||
// Use ditto mark if same time as previous row
|
||||
const timeDisplay = time === lastTime ? '″' : time;
|
||||
|
||||
return {
|
||||
row: `| ${id} | ${timeDisplay} | ${icon} | ${title} | - |`,
|
||||
time
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Format user prompt as table row for search results (no Work column)
|
||||
*/
|
||||
formatUserPromptSearchRow(prompt: UserPromptSearchResult, lastTime: string): { row: string; time: string } {
|
||||
const id = `#P${prompt.id}`;
|
||||
const time = this.formatTime(prompt.created_at_epoch);
|
||||
const icon = '💬';
|
||||
// Truncate long prompts for table display
|
||||
const title = prompt.prompt_text.length > 60
|
||||
? prompt.prompt_text.substring(0, 57) + '...'
|
||||
: prompt.prompt_text;
|
||||
|
||||
// Use ditto mark if same time as previous row
|
||||
const timeDisplay = time === lastTime ? '″' : time;
|
||||
|
||||
return {
|
||||
row: `| ${id} | ${timeDisplay} | ${icon} | ${title} | - |`,
|
||||
time
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user