Merge feature/import-export: Add memory export/import scripts with duplicate prevention
This commit is contained in:
Alex Newman
@@ -39,6 +39,7 @@
|
||||
"usage/search-tools",
|
||||
"usage/claude-desktop",
|
||||
"usage/private-tags",
|
||||
"usage/export-import",
|
||||
"beta-features"
|
||||
]
|
||||
},
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
---
|
||||
title: "Memory Export/Import"
|
||||
description: "Share knowledge across claude-mem installations with duplicate prevention"
|
||||
---
|
||||
|
||||
# Memory Export/Import Scripts
|
||||
|
||||
Share your claude-mem knowledge with other users! These scripts allow you to export specific memories (observations, sessions, summaries, and prompts) and import them into another claude-mem installation.
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **Share Windows compatibility knowledge** with Windows users
|
||||
- **Share bug fix patterns** with contributors
|
||||
- **Share project-specific learnings** across teams
|
||||
- **Backup specific memory sets** for safekeeping
|
||||
|
||||
## How It Works
|
||||
|
||||
### Export Script
|
||||
|
||||
Searches the database using **hybrid search** (combines ChromaDB vector embeddings with FTS5 full-text search) and exports all matching:
|
||||
- **Observations** - Individual learnings and discoveries
|
||||
- **Sessions** - Session metadata
|
||||
- **Summaries** - Session summaries
|
||||
- **Prompts** - User prompts that led to the work
|
||||
|
||||
Output is a portable JSON file that can be shared.
|
||||
|
||||
> **Privacy Note:** Export files contain all matching memory data in plain text. Review exports before sharing to ensure no sensitive information (API keys, passwords, private paths) is included.
|
||||
|
||||
### Import Script
|
||||
|
||||
Imports memories with **duplicate prevention**:
|
||||
- Checks if each record already exists before inserting
|
||||
- Skips duplicates automatically
|
||||
- Maintains data integrity with transactional imports
|
||||
- Reports what was imported vs. skipped
|
||||
|
||||
**Duplicate Detection Strategy:**
|
||||
- **Sessions**: By `claude_session_id` (unique)
|
||||
- **Summaries**: By `sdk_session_id` (unique)
|
||||
- **Observations**: By `sdk_session_id` + `title` + `created_at_epoch` (composite)
|
||||
- **Prompts**: By `claude_session_id` + `prompt_number` (composite)
|
||||
|
||||
## Usage
|
||||
|
||||
### Export Memories
|
||||
|
||||
```bash
|
||||
# Export all Windows-related memories
|
||||
npx tsx scripts/export-memories.ts "windows" windows-memories.json
|
||||
|
||||
# Export bug fixes
|
||||
npx tsx scripts/export-memories.ts "bugfix" bugfixes.json
|
||||
|
||||
# Export specific feature work
|
||||
npx tsx scripts/export-memories.ts "progressive disclosure" progressive-disclosure.json
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
1. `<query>` - Search query (uses hybrid semantic + full-text search)
|
||||
2. `<output-file>` - Output JSON file path
|
||||
3. `--project=name` - Optional: filter results to a specific project
|
||||
|
||||
**Example Output:**
|
||||
```
|
||||
🔍 Searching for: "windows"
|
||||
✅ Found 54 observations
|
||||
✅ Found 12 sessions
|
||||
✅ Found 12 summaries
|
||||
✅ Found 7 prompts
|
||||
|
||||
📦 Export complete!
|
||||
📄 Output: windows-memories.json
|
||||
📊 Stats:
|
||||
• 54 observations
|
||||
• 12 sessions
|
||||
• 12 summaries
|
||||
• 7 prompts
|
||||
```
|
||||
|
||||
### Import Memories
|
||||
|
||||
```bash
|
||||
# Import from an export file
|
||||
npx tsx scripts/import-memories.ts windows-memories.json
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
1. `<input-file>` - Input JSON file (from export script)
|
||||
|
||||
**Example Output:**
|
||||
```
|
||||
📦 Import file: windows-memories.json
|
||||
📅 Exported: 2025-12-10T23:45:00.000Z
|
||||
🔍 Query: "windows"
|
||||
📊 Contains:
|
||||
• 54 observations
|
||||
• 12 sessions
|
||||
• 12 summaries
|
||||
• 7 prompts
|
||||
|
||||
🔄 Importing sessions...
|
||||
✅ Imported: 12, Skipped: 0
|
||||
🔄 Importing summaries...
|
||||
✅ Imported: 12, Skipped: 0
|
||||
🔄 Importing observations...
|
||||
✅ Imported: 54, Skipped: 0
|
||||
🔄 Importing prompts...
|
||||
✅ Imported: 7, Skipped: 0
|
||||
|
||||
✅ Import complete!
|
||||
📊 Summary:
|
||||
Sessions: 12 imported, 0 skipped
|
||||
Summaries: 12 imported, 0 skipped
|
||||
Observations: 54 imported, 0 skipped
|
||||
Prompts: 7 imported, 0 skipped
|
||||
```
|
||||
|
||||
### Re-importing (Duplicate Prevention)
|
||||
|
||||
If you run the import again on the same file, duplicates are automatically skipped:
|
||||
|
||||
```
|
||||
🔄 Importing sessions...
|
||||
✅ Imported: 0, Skipped: 12 ← All skipped (already exist)
|
||||
🔄 Importing summaries...
|
||||
✅ Imported: 0, Skipped: 12
|
||||
🔄 Importing observations...
|
||||
✅ Imported: 0, Skipped: 54
|
||||
🔄 Importing prompts...
|
||||
✅ Imported: 0, Skipped: 7
|
||||
```
|
||||
|
||||
## Sharing Memories
|
||||
|
||||
### For Export Authors
|
||||
|
||||
1. **Export your memories:**
|
||||
```bash
|
||||
npx tsx scripts/export-memories.ts "windows" windows-memories.json
|
||||
```
|
||||
|
||||
2. **Share the JSON file** via:
|
||||
- GitHub gist
|
||||
- Project repository (`shared-memories/`)
|
||||
- Direct file transfer
|
||||
- Package in releases
|
||||
|
||||
3. **Document what's included:**
|
||||
- What query was used
|
||||
- What knowledge is contained
|
||||
- Who might benefit from it
|
||||
|
||||
### For Import Users
|
||||
|
||||
1. **Download the export file** to your local machine
|
||||
|
||||
2. **Review what's in it** (optional):
|
||||
```bash
|
||||
cat windows-memories.json | jq '.totalObservations, .totalSessions'
|
||||
```
|
||||
|
||||
3. **Import into your database:**
|
||||
```bash
|
||||
npx tsx scripts/import-memories.ts windows-memories.json
|
||||
```
|
||||
|
||||
4. **Verify import** by searching:
|
||||
```bash
|
||||
curl "http://localhost:37777/api/search?query=windows&format=index&limit=10"
|
||||
```
|
||||
|
||||
## JSON Export Format
|
||||
|
||||
```json
|
||||
{
|
||||
"exportedAt": "2025-12-10T23:45:00.000Z",
|
||||
"exportedAtEpoch": 1733876700000,
|
||||
"query": "windows",
|
||||
"totalObservations": 54,
|
||||
"totalSessions": 12,
|
||||
"totalSummaries": 12,
|
||||
"totalPrompts": 7,
|
||||
"observations": [ /* array of observation objects */ ],
|
||||
"sessions": [ /* array of session objects */ ],
|
||||
"summaries": [ /* array of summary objects */ ],
|
||||
"prompts": [ /* array of prompt objects */ ]
|
||||
}
|
||||
```
|
||||
|
||||
## Safety Features
|
||||
|
||||
✅ **Duplicate Prevention** - Won't re-import existing records
|
||||
✅ **Transactional** - All-or-nothing imports (database stays consistent)
|
||||
✅ **Read-only Export** - Export script opens database in read-only mode
|
||||
✅ **Dependency Ordering** - Sessions imported before observations/summaries
|
||||
✅ **Validation** - Checks database exists before starting
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Export by Project
|
||||
|
||||
```bash
|
||||
# Export only claude-mem project memories
|
||||
npx tsx scripts/export-memories.ts "bugfix" bugfixes.json --project=claude-mem
|
||||
|
||||
# Export all memories for a specific project
|
||||
npx tsx scripts/export-memories.ts "" all-project.json --project=my-app
|
||||
```
|
||||
|
||||
### Export by Type
|
||||
|
||||
```bash
|
||||
# Export only discoveries
|
||||
npx tsx scripts/export-memories.ts "type:discovery" discoveries.json
|
||||
|
||||
# Export only bug fixes
|
||||
npx tsx scripts/export-memories.ts "type:bugfix" bugfixes.json
|
||||
```
|
||||
|
||||
### Export by Date Range
|
||||
|
||||
You can filter the export after exporting:
|
||||
|
||||
```bash
|
||||
# Export all memories, then filter manually with jq
|
||||
npx tsx scripts/export-memories.ts "" all-memories.json
|
||||
cat all-memories.json | jq '.observations |= map(select(.created_at_epoch > 1700000000000))' > recent-memories.json
|
||||
```
|
||||
|
||||
### Combine Multiple Exports
|
||||
|
||||
```bash
|
||||
# Export different topics
|
||||
npx tsx scripts/export-memories.ts "windows" windows.json
|
||||
npx tsx scripts/export-memories.ts "linux" linux.json
|
||||
|
||||
# Import both
|
||||
npx tsx scripts/import-memories.ts windows.json
|
||||
npx tsx scripts/import-memories.ts linux.json
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Database Not Found
|
||||
|
||||
```
|
||||
❌ Database not found at: /Users/you/.claude-mem/claude-mem.db
|
||||
```
|
||||
|
||||
**Solution:** Make sure claude-mem is installed and has been run at least once.
|
||||
|
||||
### Import File Not Found
|
||||
|
||||
```
|
||||
❌ Input file not found: windows-memories.json
|
||||
```
|
||||
|
||||
**Solution:** Check the file path. Use absolute paths if needed.
|
||||
|
||||
### Partial Import
|
||||
|
||||
If import fails mid-way, the transaction is rolled back - your database remains unchanged. Fix the issue and try again.
|
||||
|
||||
## Contributing Memory Sets
|
||||
|
||||
If you've exported valuable knowledge that others might benefit from:
|
||||
|
||||
1. Create a PR to the `shared-memories/` directory
|
||||
2. Include a README describing what's in the export
|
||||
3. Tag with relevant keywords (windows, linux, bugfix, etc.)
|
||||
4. Community members can then import your knowledge!
|
||||
|
||||
## Examples of Useful Exports
|
||||
|
||||
**Windows Compatibility Knowledge:**
|
||||
```bash
|
||||
npx tsx scripts/export-memories.ts "windows compatibility installation" windows-fixes.json
|
||||
```
|
||||
|
||||
**Progressive Disclosure Architecture:**
|
||||
```bash
|
||||
npx tsx scripts/export-memories.ts "progressive disclosure architecture token" pd-patterns.json
|
||||
```
|
||||
|
||||
**Bug Fix Patterns:**
|
||||
```bash
|
||||
npx tsx scripts/export-memories.ts "bugfix error handling" bugfix-patterns.json
|
||||
```
|
||||
|
||||
**Performance Optimization:**
|
||||
```bash
|
||||
npx tsx scripts/export-memories.ts "performance optimization caching" perf-tips.json
|
||||
```
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,207 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Export memories matching a search query to a portable JSON format
|
||||
* Usage: npx tsx scripts/export-memories.ts <query> <output-file> [--project=name]
|
||||
* Example: npx tsx scripts/export-memories.ts "windows" windows-memories.json --project=claude-mem
|
||||
*/
|
||||
|
||||
import Database from 'better-sqlite3';
|
||||
import { existsSync, writeFileSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { SettingsDefaultsManager } from '../src/shared/SettingsDefaultsManager';
|
||||
|
||||
interface ObservationRecord {
|
||||
id: number;
|
||||
sdk_session_id: string;
|
||||
project: string;
|
||||
text: string | null;
|
||||
type: string;
|
||||
title: string;
|
||||
subtitle: string | null;
|
||||
facts: string | null;
|
||||
narrative: string | null;
|
||||
concepts: string | null;
|
||||
files_read: string | null;
|
||||
files_modified: string | null;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number | null;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
interface SdkSessionRecord {
|
||||
id: number;
|
||||
claude_session_id: string;
|
||||
sdk_session_id: string;
|
||||
project: string;
|
||||
user_prompt: string;
|
||||
started_at: string;
|
||||
started_at_epoch: number;
|
||||
completed_at: string | null;
|
||||
completed_at_epoch: number | null;
|
||||
status: string;
|
||||
}
|
||||
|
||||
interface SessionSummaryRecord {
|
||||
id: number;
|
||||
sdk_session_id: string;
|
||||
project: string;
|
||||
request: string | null;
|
||||
investigated: string | null;
|
||||
learned: string | null;
|
||||
completed: string | null;
|
||||
next_steps: string | null;
|
||||
files_read: string | null;
|
||||
files_edited: string | null;
|
||||
notes: string | null;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number | null;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
interface UserPromptRecord {
|
||||
id: number;
|
||||
claude_session_id: string;
|
||||
prompt_number: number;
|
||||
prompt_text: string;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
interface ExportData {
|
||||
exportedAt: string;
|
||||
exportedAtEpoch: number;
|
||||
query: string;
|
||||
project?: string;
|
||||
totalObservations: number;
|
||||
totalSessions: number;
|
||||
totalSummaries: number;
|
||||
totalPrompts: number;
|
||||
observations: ObservationRecord[];
|
||||
sessions: SdkSessionRecord[];
|
||||
summaries: SessionSummaryRecord[];
|
||||
prompts: UserPromptRecord[];
|
||||
}
|
||||
|
||||
async function exportMemories(query: string, outputFile: string, project?: string) {
|
||||
try {
|
||||
// Read port from settings
|
||||
const settings = SettingsDefaultsManager.loadFromFile(join(homedir(), '.claude-mem', 'settings.json'));
|
||||
const port = parseInt(settings.CLAUDE_MEM_WORKER_PORT, 10);
|
||||
const baseUrl = `http://localhost:${port}`;
|
||||
|
||||
console.log(`🔍 Searching for: "${query}"${project ? ` (project: ${project})` : ' (all projects)'}`);
|
||||
|
||||
// Build query params - use format=json for raw data
|
||||
const params = new URLSearchParams({
|
||||
query,
|
||||
format: 'json',
|
||||
limit: '999999'
|
||||
});
|
||||
if (project) params.set('project', project);
|
||||
|
||||
// Unified search - gets all result types using hybrid search
|
||||
console.log('📡 Fetching all memories via hybrid search...');
|
||||
const searchResponse = await fetch(`${baseUrl}/api/search?${params.toString()}`);
|
||||
if (!searchResponse.ok) {
|
||||
throw new Error(`Failed to search: ${searchResponse.status} ${searchResponse.statusText}`);
|
||||
}
|
||||
const searchData = await searchResponse.json();
|
||||
|
||||
const observations: ObservationRecord[] = searchData.observations || [];
|
||||
const summaries: SessionSummaryRecord[] = searchData.sessions || [];
|
||||
const prompts: UserPromptRecord[] = searchData.prompts || [];
|
||||
|
||||
console.log(`✅ Found ${observations.length} observations`);
|
||||
console.log(`✅ Found ${summaries.length} session summaries`);
|
||||
console.log(`✅ Found ${prompts.length} user prompts`);
|
||||
|
||||
// Get unique SDK session IDs from observations and summaries
|
||||
const sdkSessionIds = new Set<string>();
|
||||
observations.forEach((o) => {
|
||||
if (o.sdk_session_id) sdkSessionIds.add(o.sdk_session_id);
|
||||
});
|
||||
summaries.forEach((s) => {
|
||||
if (s.sdk_session_id) sdkSessionIds.add(s.sdk_session_id);
|
||||
});
|
||||
|
||||
// Get SDK sessions metadata from database
|
||||
// (We need this because the API doesn't expose sdk_sessions table directly)
|
||||
console.log('📡 Fetching SDK sessions metadata...');
|
||||
const sessions: SdkSessionRecord[] = [];
|
||||
if (sdkSessionIds.size > 0) {
|
||||
// Read directly from database for sdk_sessions table
|
||||
const Database = (await import('better-sqlite3')).default;
|
||||
const dbPath = join(homedir(), '.claude-mem', 'claude-mem.db');
|
||||
|
||||
if (!existsSync(dbPath)) {
|
||||
console.error(`❌ Database not found at: ${dbPath}`);
|
||||
console.error('💡 Has claude-mem been initialized? Try running a session first.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const db = new Database(dbPath, { readonly: true });
|
||||
|
||||
try {
|
||||
const placeholders = Array.from(sdkSessionIds).map(() => '?').join(',');
|
||||
const sessionQuery = `
|
||||
SELECT * FROM sdk_sessions
|
||||
WHERE sdk_session_id IN (${placeholders})
|
||||
ORDER BY started_at_epoch DESC
|
||||
`;
|
||||
sessions.push(...db.prepare(sessionQuery).all(...Array.from(sdkSessionIds)));
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
console.log(`✅ Found ${sessions.length} SDK sessions`);
|
||||
|
||||
// Create export data
|
||||
const exportData: ExportData = {
|
||||
exportedAt: new Date().toISOString(),
|
||||
exportedAtEpoch: Date.now(),
|
||||
query,
|
||||
project,
|
||||
totalObservations: observations.length,
|
||||
totalSessions: sessions.length,
|
||||
totalSummaries: summaries.length,
|
||||
totalPrompts: prompts.length,
|
||||
observations,
|
||||
sessions,
|
||||
summaries,
|
||||
prompts
|
||||
};
|
||||
|
||||
// Write to file
|
||||
writeFileSync(outputFile, JSON.stringify(exportData, null, 2));
|
||||
|
||||
console.log(`\n📦 Export complete!`);
|
||||
console.log(`📄 Output: ${outputFile}`);
|
||||
console.log(`📊 Stats:`);
|
||||
console.log(` • ${exportData.totalObservations} observations`);
|
||||
console.log(` • ${exportData.totalSessions} sessions`);
|
||||
console.log(` • ${exportData.totalSummaries} summaries`);
|
||||
console.log(` • ${exportData.totalPrompts} prompts`);
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Export failed:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// CLI interface
|
||||
const args = process.argv.slice(2);
|
||||
if (args.length < 2) {
|
||||
console.error('Usage: npx tsx scripts/export-memories.ts <query> <output-file> [--project=name]');
|
||||
console.error('Example: npx tsx scripts/export-memories.ts "windows" windows-memories.json --project=claude-mem');
|
||||
console.error(' npx tsx scripts/export-memories.ts "authentication" auth.json');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Parse arguments
|
||||
const [query, outputFile, ...flags] = args;
|
||||
const project = flags.find(f => f.startsWith('--project='))?.split('=')[1];
|
||||
|
||||
exportMemories(query, outputFile, project);
|
||||
@@ -0,0 +1,245 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Import memories from a JSON export file with duplicate prevention
|
||||
* Usage: npx tsx scripts/import-memories.ts <input-file>
|
||||
* Example: npx tsx scripts/import-memories.ts windows-memories.json
|
||||
*/
|
||||
|
||||
import Database from 'better-sqlite3';
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
interface ImportStats {
|
||||
sessionsImported: number;
|
||||
sessionsSkipped: number;
|
||||
summariesImported: number;
|
||||
summariesSkipped: number;
|
||||
observationsImported: number;
|
||||
observationsSkipped: number;
|
||||
promptsImported: number;
|
||||
promptsSkipped: number;
|
||||
}
|
||||
|
||||
function importMemories(inputFile: string) {
|
||||
if (!existsSync(inputFile)) {
|
||||
console.error(`❌ Input file not found: ${inputFile}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const dbPath = join(homedir(), '.claude-mem', 'claude-mem.db');
|
||||
|
||||
if (!existsSync(dbPath)) {
|
||||
console.error(`❌ Database not found at: ${dbPath}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Read and parse export file
|
||||
const exportData = JSON.parse(readFileSync(inputFile, 'utf-8'));
|
||||
|
||||
console.log(`📦 Import file: ${inputFile}`);
|
||||
console.log(`📅 Exported: ${exportData.exportedAt}`);
|
||||
console.log(`🔍 Query: "${exportData.query}"`);
|
||||
console.log(`📊 Contains:`);
|
||||
console.log(` • ${exportData.totalObservations} observations`);
|
||||
console.log(` • ${exportData.totalSessions} sessions`);
|
||||
console.log(` • ${exportData.totalSummaries} summaries`);
|
||||
console.log(` • ${exportData.totalPrompts} prompts`);
|
||||
console.log('');
|
||||
|
||||
const db = new Database(dbPath);
|
||||
const stats: ImportStats = {
|
||||
sessionsImported: 0,
|
||||
sessionsSkipped: 0,
|
||||
summariesImported: 0,
|
||||
summariesSkipped: 0,
|
||||
observationsImported: 0,
|
||||
observationsSkipped: 0,
|
||||
promptsImported: 0,
|
||||
promptsSkipped: 0
|
||||
};
|
||||
|
||||
try {
|
||||
// Prepare statements for duplicate checking
|
||||
const checkSession = db.prepare('SELECT id FROM sdk_sessions WHERE claude_session_id = ?');
|
||||
const checkSummary = db.prepare('SELECT id FROM session_summaries WHERE sdk_session_id = ?');
|
||||
const checkObservation = db.prepare(`
|
||||
SELECT id FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
AND title = ?
|
||||
AND created_at_epoch = ?
|
||||
`);
|
||||
const checkPrompt = db.prepare(`
|
||||
SELECT id FROM user_prompts
|
||||
WHERE claude_session_id = ?
|
||||
AND prompt_number = ?
|
||||
`);
|
||||
|
||||
// Prepare insert statements
|
||||
const insertSession = db.prepare(`
|
||||
INSERT INTO sdk_sessions (
|
||||
claude_session_id, sdk_session_id, project, user_prompt,
|
||||
started_at, started_at_epoch, completed_at, completed_at_epoch,
|
||||
status
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const insertSummary = db.prepare(`
|
||||
INSERT INTO session_summaries (
|
||||
sdk_session_id, project, request, investigated, learned,
|
||||
completed, next_steps, files_read, files_edited, notes,
|
||||
prompt_number, discovery_tokens, created_at, created_at_epoch
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const insertObservation = db.prepare(`
|
||||
INSERT INTO observations (
|
||||
sdk_session_id, project, text, type, title, subtitle,
|
||||
facts, narrative, concepts, files_read, files_modified,
|
||||
prompt_number, discovery_tokens, created_at, created_at_epoch
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const insertPrompt = db.prepare(`
|
||||
INSERT INTO user_prompts (
|
||||
claude_session_id, prompt_number, prompt_text,
|
||||
created_at, created_at_epoch
|
||||
) VALUES (?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
// Import in transaction
|
||||
db.transaction(() => {
|
||||
// 1. Import sessions first (dependency for everything else)
|
||||
console.log('🔄 Importing sessions...');
|
||||
for (const session of exportData.sessions) {
|
||||
const exists = checkSession.get(session.claude_session_id);
|
||||
if (exists) {
|
||||
stats.sessionsSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
insertSession.run(
|
||||
session.claude_session_id,
|
||||
session.sdk_session_id,
|
||||
session.project,
|
||||
session.user_prompt,
|
||||
session.started_at,
|
||||
session.started_at_epoch,
|
||||
session.completed_at,
|
||||
session.completed_at_epoch,
|
||||
session.status
|
||||
);
|
||||
stats.sessionsImported++;
|
||||
}
|
||||
console.log(` ✅ Imported: ${stats.sessionsImported}, Skipped: ${stats.sessionsSkipped}`);
|
||||
|
||||
// 2. Import summaries (depends on sessions)
|
||||
console.log('🔄 Importing summaries...');
|
||||
for (const summary of exportData.summaries) {
|
||||
const exists = checkSummary.get(summary.sdk_session_id);
|
||||
if (exists) {
|
||||
stats.summariesSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
insertSummary.run(
|
||||
summary.sdk_session_id,
|
||||
summary.project,
|
||||
summary.request,
|
||||
summary.investigated,
|
||||
summary.learned,
|
||||
summary.completed,
|
||||
summary.next_steps,
|
||||
summary.files_read,
|
||||
summary.files_edited,
|
||||
summary.notes,
|
||||
summary.prompt_number,
|
||||
summary.discovery_tokens || 0,
|
||||
summary.created_at,
|
||||
summary.created_at_epoch
|
||||
);
|
||||
stats.summariesImported++;
|
||||
}
|
||||
console.log(` ✅ Imported: ${stats.summariesImported}, Skipped: ${stats.summariesSkipped}`);
|
||||
|
||||
// 3. Import observations (depends on sessions)
|
||||
console.log('🔄 Importing observations...');
|
||||
for (const obs of exportData.observations) {
|
||||
const exists = checkObservation.get(
|
||||
obs.sdk_session_id,
|
||||
obs.title,
|
||||
obs.created_at_epoch
|
||||
);
|
||||
if (exists) {
|
||||
stats.observationsSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
insertObservation.run(
|
||||
obs.sdk_session_id,
|
||||
obs.project,
|
||||
obs.text,
|
||||
obs.type,
|
||||
obs.title,
|
||||
obs.subtitle,
|
||||
obs.facts,
|
||||
obs.narrative,
|
||||
obs.concepts,
|
||||
obs.files_read,
|
||||
obs.files_modified,
|
||||
obs.prompt_number,
|
||||
obs.discovery_tokens || 0,
|
||||
obs.created_at,
|
||||
obs.created_at_epoch
|
||||
);
|
||||
stats.observationsImported++;
|
||||
}
|
||||
console.log(` ✅ Imported: ${stats.observationsImported}, Skipped: ${stats.observationsSkipped}`);
|
||||
|
||||
// 4. Import prompts (depends on sessions)
|
||||
console.log('🔄 Importing prompts...');
|
||||
for (const prompt of exportData.prompts) {
|
||||
const exists = checkPrompt.get(
|
||||
prompt.claude_session_id,
|
||||
prompt.prompt_number
|
||||
);
|
||||
if (exists) {
|
||||
stats.promptsSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
insertPrompt.run(
|
||||
prompt.claude_session_id,
|
||||
prompt.prompt_number,
|
||||
prompt.prompt_text,
|
||||
prompt.created_at,
|
||||
prompt.created_at_epoch
|
||||
);
|
||||
stats.promptsImported++;
|
||||
}
|
||||
console.log(` ✅ Imported: ${stats.promptsImported}, Skipped: ${stats.promptsSkipped}`);
|
||||
|
||||
})();
|
||||
|
||||
console.log('\n✅ Import complete!');
|
||||
console.log('📊 Summary:');
|
||||
console.log(` Sessions: ${stats.sessionsImported} imported, ${stats.sessionsSkipped} skipped`);
|
||||
console.log(` Summaries: ${stats.summariesImported} imported, ${stats.summariesSkipped} skipped`);
|
||||
console.log(` Observations: ${stats.observationsImported} imported, ${stats.observationsSkipped} skipped`);
|
||||
console.log(` Prompts: ${stats.promptsImported} imported, ${stats.promptsSkipped} skipped`);
|
||||
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
// CLI interface
|
||||
const args = process.argv.slice(2);
|
||||
if (args.length < 1) {
|
||||
console.error('Usage: npx tsx scripts/import-memories.ts <input-file>');
|
||||
console.error('Example: npx tsx scripts/import-memories.ts windows-memories.json');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const [inputFile] = args;
|
||||
importMemories(inputFile);
|
||||
@@ -811,26 +811,72 @@ export class SessionStore {
|
||||
*/
|
||||
getObservationsByIds(
|
||||
ids: number[],
|
||||
options: { orderBy?: 'date_desc' | 'date_asc'; limit?: number } = {}
|
||||
options: { orderBy?: 'date_desc' | 'date_asc'; limit?: number; project?: string; type?: string | string[]; concepts?: string | string[]; files?: string | string[] } = {}
|
||||
): ObservationRecord[] {
|
||||
if (ids.length === 0) return [];
|
||||
|
||||
const { orderBy = 'date_desc', limit } = options;
|
||||
const { orderBy = 'date_desc', limit, project, type, concepts, files } = options;
|
||||
const orderClause = orderBy === 'date_asc' ? 'ASC' : 'DESC';
|
||||
const limitClause = limit ? `LIMIT ${limit}` : '';
|
||||
|
||||
// Build placeholders for IN clause
|
||||
const placeholders = ids.map(() => '?').join(',');
|
||||
const params: any[] = [...ids];
|
||||
const additionalConditions: string[] = [];
|
||||
|
||||
// Apply project filter
|
||||
if (project) {
|
||||
additionalConditions.push('project = ?');
|
||||
params.push(project);
|
||||
}
|
||||
|
||||
// Apply type filter
|
||||
if (type) {
|
||||
if (Array.isArray(type)) {
|
||||
const typePlaceholders = type.map(() => '?').join(',');
|
||||
additionalConditions.push(`type IN (${typePlaceholders})`);
|
||||
params.push(...type);
|
||||
} else {
|
||||
additionalConditions.push('type = ?');
|
||||
params.push(type);
|
||||
}
|
||||
}
|
||||
|
||||
// Apply concepts filter
|
||||
if (concepts) {
|
||||
const conceptsList = Array.isArray(concepts) ? concepts : [concepts];
|
||||
const conceptConditions = conceptsList.map(() =>
|
||||
'EXISTS (SELECT 1 FROM json_each(concepts) WHERE value = ?)'
|
||||
);
|
||||
params.push(...conceptsList);
|
||||
additionalConditions.push(`(${conceptConditions.join(' OR ')})`);
|
||||
}
|
||||
|
||||
// Apply files filter
|
||||
if (files) {
|
||||
const filesList = Array.isArray(files) ? files : [files];
|
||||
const fileConditions = filesList.map(() => {
|
||||
return '(EXISTS (SELECT 1 FROM json_each(files_read) WHERE value LIKE ?) OR EXISTS (SELECT 1 FROM json_each(files_modified) WHERE value LIKE ?))';
|
||||
});
|
||||
filesList.forEach(file => {
|
||||
params.push(`%${file}%`, `%${file}%`);
|
||||
});
|
||||
additionalConditions.push(`(${fileConditions.join(' OR ')})`);
|
||||
}
|
||||
|
||||
const whereClause = additionalConditions.length > 0
|
||||
? `WHERE id IN (${placeholders}) AND ${additionalConditions.join(' AND ')}`
|
||||
: `WHERE id IN (${placeholders})`;
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id IN (${placeholders})
|
||||
${whereClause}
|
||||
ORDER BY created_at_epoch ${orderClause}
|
||||
${limitClause}
|
||||
`);
|
||||
|
||||
return stmt.all(...ids) as ObservationRecord[];
|
||||
return stmt.all(...params) as ObservationRecord[];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1353,23 +1399,30 @@ export class SessionStore {
|
||||
*/
|
||||
getSessionSummariesByIds(
|
||||
ids: number[],
|
||||
options: { orderBy?: 'date_desc' | 'date_asc'; limit?: number } = {}
|
||||
options: { orderBy?: 'date_desc' | 'date_asc'; limit?: number; project?: string } = {}
|
||||
): SessionSummaryRecord[] {
|
||||
if (ids.length === 0) return [];
|
||||
|
||||
const { orderBy = 'date_desc', limit } = options;
|
||||
const { orderBy = 'date_desc', limit, project } = options;
|
||||
const orderClause = orderBy === 'date_asc' ? 'ASC' : 'DESC';
|
||||
const limitClause = limit ? `LIMIT ${limit}` : '';
|
||||
const placeholders = ids.map(() => '?').join(',');
|
||||
const params: any[] = [...ids];
|
||||
|
||||
// Apply project filter
|
||||
const whereClause = project
|
||||
? `WHERE id IN (${placeholders}) AND project = ?`
|
||||
: `WHERE id IN (${placeholders})`;
|
||||
if (project) params.push(project);
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT * FROM session_summaries
|
||||
WHERE id IN (${placeholders})
|
||||
${whereClause}
|
||||
ORDER BY created_at_epoch ${orderClause}
|
||||
${limitClause}
|
||||
`);
|
||||
|
||||
return stmt.all(...ids) as SessionSummaryRecord[];
|
||||
return stmt.all(...params) as SessionSummaryRecord[];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1378,14 +1431,19 @@ export class SessionStore {
|
||||
*/
|
||||
getUserPromptsByIds(
|
||||
ids: number[],
|
||||
options: { orderBy?: 'date_desc' | 'date_asc'; limit?: number } = {}
|
||||
options: { orderBy?: 'date_desc' | 'date_asc'; limit?: number; project?: string } = {}
|
||||
): UserPromptRecord[] {
|
||||
if (ids.length === 0) return [];
|
||||
|
||||
const { orderBy = 'date_desc', limit } = options;
|
||||
const { orderBy = 'date_desc', limit, project } = options;
|
||||
const orderClause = orderBy === 'date_asc' ? 'ASC' : 'DESC';
|
||||
const limitClause = limit ? `LIMIT ${limit}` : '';
|
||||
const placeholders = ids.map(() => '?').join(',');
|
||||
const params: any[] = [...ids];
|
||||
|
||||
// Apply project filter
|
||||
const projectFilter = project ? 'AND s.project = ?' : '';
|
||||
if (project) params.push(project);
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT
|
||||
@@ -1394,12 +1452,12 @@ export class SessionStore {
|
||||
s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.id IN (${placeholders})
|
||||
WHERE up.id IN (${placeholders}) ${projectFilter}
|
||||
ORDER BY up.created_at_epoch ${orderClause}
|
||||
${limitClause}
|
||||
`);
|
||||
|
||||
return stmt.all(...ids) as UserPromptRecord[];
|
||||
return stmt.all(...params) as UserPromptRecord[];
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -166,10 +166,10 @@ export class SearchManager {
|
||||
observations = this.sessionStore.getObservationsByIds(obsIds, obsOptions);
|
||||
}
|
||||
if (sessionIds.length > 0) {
|
||||
sessions = this.sessionStore.getSessionSummariesByIds(sessionIds, { orderBy: 'date_desc', limit: options.limit });
|
||||
sessions = this.sessionStore.getSessionSummariesByIds(sessionIds, { orderBy: 'date_desc', limit: options.limit, project: options.project });
|
||||
}
|
||||
if (promptIds.length > 0) {
|
||||
prompts = this.sessionStore.getUserPromptsByIds(promptIds, { orderBy: 'date_desc', limit: options.limit });
|
||||
prompts = this.sessionStore.getUserPromptsByIds(promptIds, { orderBy: 'date_desc', limit: options.limit, project: options.project });
|
||||
}
|
||||
|
||||
logger.debug('SEARCH', 'Hydrated results from SQLite', { observations: observations.length, sessions: sessions.length, prompts: prompts.length });
|
||||
@@ -198,6 +198,13 @@ export class SearchManager {
|
||||
const totalResults = observations.length + sessions.length + prompts.length;
|
||||
|
||||
if (totalResults === 0) {
|
||||
if (format === 'json') {
|
||||
return {
|
||||
observations: [],
|
||||
sessions: [],
|
||||
prompts: []
|
||||
};
|
||||
}
|
||||
return {
|
||||
content: [{
|
||||
type: 'text' as const,
|
||||
@@ -230,6 +237,15 @@ export class SearchManager {
|
||||
const limitedResults = allResults.slice(0, options.limit || 20);
|
||||
|
||||
// Format based on requested format
|
||||
if (format === 'json') {
|
||||
// Raw JSON format for exports
|
||||
return {
|
||||
observations,
|
||||
sessions,
|
||||
prompts
|
||||
};
|
||||
}
|
||||
|
||||
let combinedText: string;
|
||||
if (format === 'index') {
|
||||
const header = `Found ${totalResults} result(s) matching "${query}" (${observations.length} obs, ${sessions.length} sessions, ${prompts.length} prompts):\n\n`;
|
||||
|
||||
Reference in New Issue
Block a user