feat: Mode system with inheritance and multilingual support (#412)

* feat: add domain management system with support for multiple domain profiles

- Introduced DomainManager class for loading and managing domain profiles.
- Added support for a default domain ('code') and fallback mechanisms.
- Implemented domain configuration validation and error handling.
- Created types for domain configuration, observation types, and concepts.
- Added new directory for domain profiles and ensured its existence.
- Updated SettingsDefaultsManager to include CLAUDE_MEM_DOMAIN setting.

* Refactor domain management to mode management

- Removed DomainManager class and replaced it with ModeManager for better clarity and functionality.
- Updated types from DomainConfig to ModeConfig and DomainPrompts to ModePrompts.
- Changed references from domains to modes in the settings and paths.
- Ensured backward compatibility by maintaining the fallback mechanism to the 'code' mode.

* feat: add migration 008 to support mode-agnostic observations and refactor service layer references in documentation

* feat: add new modes for code development and email investigation with detailed observation types and concepts

* Refactor observation parsing and prompt generation to incorporate mode-specific configurations

- Updated `parseObservations` function to use 'observation' as a universal fallback type instead of 'change', utilizing active mode's valid observation types.
- Modified `buildInitPrompt` and `buildContinuationPrompt` functions to accept a `ModeConfig` parameter, allowing for dynamic prompt content based on the active mode.
- Enhanced `ModePrompts` interface to include additional guidance for observers, such as recording focus and skip guidance.
- Adjusted the SDKAgent to load the active mode and pass it to prompt generation functions, ensuring prompts are tailored to the current mode's context.

* fix: correct mode prompt injection to preserve exact wording and type list visibility

- Add script to extract prompts from main branch prompts.ts into code.yaml
- Fix prompts.ts to show type list in XML template (e.g., "[ bugfix | feature | ... ]")
- Keep 'change' as fallback type in parser.ts (maintain backwards compatibility)
- Regenerate code.yaml with exact wording from original hardcoded prompts
- Build succeeds with no TypeScript errors

* fix: update ModeManager to load JSON mode files and improve validation

- Changed ModeManager to load mode configurations from JSON files instead of YAML.
- Removed the requirement for an "observation" type and updated validation to require at least one observation type.
- Updated fallback behavior in the parser to use the first type from the active mode's type list.
- Added comprehensive tests for mode loading, prompt injection, and parser integration, ensuring correct behavior across different modes.
- Introduced new mode JSON files for "Code Development" and "Email Investigation" with detailed observation types and prompts.

* Add mode configuration loading and update licensing information for Ragtime

- Implemented loading of mode configuration in WorkerService before database initialization.
- Added PolyForm Noncommercial License 1.0.0 to Ragtime directory.
- Created README.md for Ragtime with licensing details and usage guidelines.

* fix: add datasets directory to .gitignore to prevent accidental commits

* refactor: remove unused plugin package.json file

* chore: add package.json for claude-mem plugin with version 7.4.5

* refactor: remove outdated tests and improve error handling

- Deleted tests for ChromaSync error handling, smart install, strip memory tags, and user prompt tag stripping due to redundancy or outdated logic.
- Removed vitest configuration as it is no longer needed.
- Added a comprehensive implementation plan for fixing the modes system, addressing critical issues and improving functionality.
- Created a detailed test analysis report highlighting the quality and effectiveness of the current test suite, identifying areas for improvement.
- Introduced a new plugin package.json for runtime dependencies related to claude-mem hooks.

* refactor: remove parser regression tests to streamline codebase

* docs: update CLAUDE.md to clarify test management and changelog generation

* refactor: remove migration008 for mode-agnostic observations

* Refactor observation type handling to use ModeManager for icons and emojis

- Removed direct mappings of observation types to icons and work emojis in context-generator, FormattingService, SearchManager, and TimelineService.
- Integrated ModeManager to dynamically retrieve icons and emojis based on the active mode.
- Improved maintainability by centralizing the logic for observation type representation.

* Refactor observation metadata constants and update context generator

- Removed the explicit declaration of OBSERVATION_TYPES and OBSERVATION_CONCEPTS from observation-metadata.ts.
- Introduced fallback default strings for DEFAULT_OBSERVATION_TYPES_STRING and DEFAULT_OBSERVATION_CONCEPTS_STRING.
- Updated context-generator.ts to utilize observation types and concepts from ModeManager instead of constants.

* refactor: remove intermediate error handling from hooks (Phase 1)

Apply "fail fast" error handling strategy - errors propagate and crash loud
instead of being caught, wrapped, and re-thrown at intermediate layers.

Changes:
- Remove try/catch around fetch calls in all hooks - let errors throw
- Add try/catch ONLY around JSON.parse at entry points
- Delete error-handler.ts and hook-error-handler.ts (no longer needed)
- Update worker-utils.ts: functions now throw instead of returning null
- Update transcript-parser.ts: throw on missing path, empty file, malformed JSON
- Remove all handleWorkerError, handleFetchError imports

Philosophy: If something breaks, we KNOW it broke. No silent failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove intermediate error handling from worker service (Phase 2)

Apply "fail fast" error handling strategy to worker service layer.

Changes:
- worker-service.ts: Remove try/catch from version endpoint, cleanup,
  MCP close, process enumeration, force kill, and isAlive check
- SessionRoutes.ts: Remove try/catch from JSON.stringify calls, remove
  .catch() from Chroma sync and SDK agent calls
- SettingsRoutes.ts: Remove try/catch from toggleMcp()
- DatabaseManager.ts: Remove .catch() from backfill and close operations
- SDKAgent.ts: Keep outer try/catch (top-level), remove .catch() from
  Chroma sync operations
- SSEBroadcaster.ts: Remove try/catch from broadcast and sendToClient

Philosophy: Errors propagate and crash loud. BaseRouteHandler.wrapHandler
provides top-level catching for HTTP routes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove error swallowing from SQLite services (Phase 3)

Apply "fail fast" error handling strategy to database layer.

SessionStore.ts migrations:
- ensureWorkerPortColumn(): Remove outer try/catch, let it throw
- ensurePromptTrackingColumns(): Remove outer try/catch, let it throw
- removeSessionSummariesUniqueConstraint(): Keep inner transaction
  rollback, remove outer catch
- addObservationHierarchicalFields(): Remove outer try/catch
- makeObservationsTextNullable(): Keep inner transaction rollback,
  remove outer catch
- createUserPromptsTable(): Keep inner transaction rollback, remove
  outer catch
- getFilesForSession(): Remove try/catch around JSON.parse

SessionSearch.ts:
- ensureFTSTables(): Remove try/catch, let it throw

Philosophy: Migration errors that are swallowed mean we think the
database is fine when it's not. Keep only inner transaction rollback
try/catch blocks.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove error hiding from utilities (Phase 4)

Apply "fail fast" error handling strategy to utility layer.

logger.ts:
- formatTool(): Remove try/catch, let JSON.parse throw on malformed input

context-generator.ts:
- loadContextConfig(): Remove try/catch, let parseInt throw on invalid settings
- Transcript extraction: Remove try/catch, let file read errors propagate

ChromaSync.ts:
- close(): Remove nested try/catch blocks, let close errors propagate

Philosophy: No silent fallbacks or hidden defaults. If something breaks,
we know it broke immediately.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: serve static UI assets and update package root path

- Added middleware to serve static UI assets (JS, CSS, fonts, etc.) in ViewerRoutes.
- Updated getPackageRoot function to correctly return the package root directory as one level up from the current directory.

* feat: Enhance mode loading with inheritance support

- Introduced parseInheritance method to handle parent--override mode IDs.
- Added deepMerge method for recursively merging mode configurations.
- Updated loadMode method to support inheritance, loading parent modes and applying overrides.
- Improved error handling for missing mode files and logging for better traceability.

* fix(modes): correct inheritance file resolution and path handling

* Refactor code structure for improved readability and maintainability

* feat: Add mode configuration documentation and examples

* fix: Improve concurrency handling in translateReadme function

* Refactor SDK prompts to enhance clarity and structure

- Updated the `buildInitPrompt` and `buildContinuationPrompt` functions in `prompts.ts` to improve the organization of prompt components, including the addition of language instructions and footer messages.
- Removed redundant instructions and emphasized the importance of recording observations.
- Modified the `ModePrompts` interface in `types.ts` to include new properties for system identity, language instructions, and output format header, ensuring better flexibility and clarity in prompt generation.

* Enhance prompts with language instructions and XML formatting

- Updated `buildInitPrompt`, `buildSummaryPrompt`, and `buildContinuationPrompt` functions to include detailed language instructions in XML comments.
- Ensured that language instructions guide users to keep XML tags in English while writing content in the specified language.
- Modified the `buildSummaryPrompt` function to accept `mode` as a parameter for consistency.
- Adjusted the call to `buildSummaryPrompt` in `SDKAgent` to pass the `mode` argument.

* Refactor XML prompt generation in SDK

- Updated the buildInitPrompt, buildSummaryPrompt, and buildContinuationPrompt functions to use new placeholders for XML elements, improving maintainability and readability.
- Removed redundant language instructions in comments for clarity.
- Added new properties to ModePrompts interface for better structure and organization of XML placeholders and section headers.

* feat: Update observation prompts and structure across multiple languages

* chore: Remove planning docs and update Ragtime README

Remove ephemeral development artifacts:
- .claude/plans/modes-system-fixes.md
- .claude/test-analysis-report.md
- PROMPT_INJECTION_ANALYSIS.md

Update ragtime/README.md to explain:
- Feature is not yet implemented
- Dependency on modes system (now complete in PR #412)
- Ready to be scripted out in future release

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: Move summary prompts to mode files for multilingual support

Summary prompts were hardcoded in English in prompts.ts, breaking
multilingual support. Now properly mode-based:

- Added summary_instruction, summary_context_label,
  summary_format_instruction, summary_footer to code.json
- Updated buildSummaryPrompt() to use mode fields instead of hardcoded text
- Added summary_footer with language instructions to all 10 language modes
- Language modes keep English prompts + language requirement footer

This fixes the gaslighting where we claimed full multilingual support
but summaries were still generated in English.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* chore: Clean up README by removing local preview instructions and streamlining beta features section

* Add translated README files for Ukrainian, Vietnamese, and Chinese languages

* Add new language modes for code development in multiple languages

- Introduced JSON configurations for Code Development in Greek, Finnish, Hebrew, Hindi, Hungarian, Indonesian, Italian, Dutch, Norwegian, Polish, Brazilian Portuguese, Romanian, Swedish, Turkish, and Ukrainian.
- Each configuration includes prompts for observations, summaries, and instructions tailored to the respective language.
- Ensured that all prompts emphasize the importance of generating observations without referencing the agent's actions.

* Add multilingual support links to README files in various languages

- Updated README.id.md, README.it.md, README.ja.md, README.ko.md, README.nl.md, README.no.md, README.pl.md, README.pt-br.md, README.ro.md, README.ru.md, README.sv.md, README.th.md, README.tr.md, README.uk.md, README.vi.md, and README.zh.md to include links to other language versions.
- Each README now features a centered paragraph with flags and links for easy navigation between different language documents.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Alex Newman
2025-12-22 20:14:18 -05:00
committed by GitHub
parent db02da148f
commit 3ea0b60b9f
141 changed files with 11834 additions and 6699 deletions
+92 -96
View File
@@ -50,104 +50,100 @@ export class SessionSearch {
* TODO: Remove FTS5 infrastructure in future major version (v7.0.0)
*/
private ensureFTSTables(): void {
try {
// Check if FTS tables already exist
const tables = this.db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name LIKE '%_fts'").all() as TableNameRow[];
const hasFTS = tables.some(t => t.name === 'observations_fts' || t.name === 'session_summaries_fts');
// Check if FTS tables already exist
const tables = this.db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name LIKE '%_fts'").all() as TableNameRow[];
const hasFTS = tables.some(t => t.name === 'observations_fts' || t.name === 'session_summaries_fts');
if (hasFTS) {
// Already migrated
return;
}
console.log('[SessionSearch] Creating FTS5 tables...');
// Create observations_fts virtual table
this.db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS observations_fts USING fts5(
title,
subtitle,
narrative,
text,
facts,
concepts,
content='observations',
content_rowid='id'
);
`);
// Populate with existing data
this.db.run(`
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
SELECT id, title, subtitle, narrative, text, facts, concepts
FROM observations;
`);
// Create triggers for observations
this.db.run(`
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
`);
// Create session_summaries_fts virtual table
this.db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS session_summaries_fts USING fts5(
request,
investigated,
learned,
completed,
next_steps,
notes,
content='session_summaries',
content_rowid='id'
);
`);
// Populate with existing data
this.db.run(`
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
SELECT id, request, investigated, learned, completed, next_steps, notes
FROM session_summaries;
`);
// Create triggers for session_summaries
this.db.run(`
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
`);
console.log('[SessionSearch] FTS5 tables created successfully');
} catch (error: any) {
console.error('[SessionSearch] FTS migration error:', error.message);
if (hasFTS) {
// Already migrated
return;
}
console.log('[SessionSearch] Creating FTS5 tables...');
// Create observations_fts virtual table
this.db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS observations_fts USING fts5(
title,
subtitle,
narrative,
text,
facts,
concepts,
content='observations',
content_rowid='id'
);
`);
// Populate with existing data
this.db.run(`
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
SELECT id, title, subtitle, narrative, text, facts, concepts
FROM observations;
`);
// Create triggers for observations
this.db.run(`
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
`);
// Create session_summaries_fts virtual table
this.db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS session_summaries_fts USING fts5(
request,
investigated,
learned,
completed,
next_steps,
notes,
content='session_summaries',
content_rowid='id'
);
`);
// Populate with existing data
this.db.run(`
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
SELECT id, request, investigated, learned, completed, next_steps, notes
FROM session_summaries;
`);
// Create triggers for session_summaries
this.db.run(`
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
`);
console.log('[SessionSearch] FTS5 tables created successfully');
}
+273 -305
View File
@@ -144,152 +144,140 @@ export class SessionStore {
* Ensure worker_port column exists (migration 5)
*/
private ensureWorkerPortColumn(): void {
try {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(5) as SchemaVersion | undefined;
if (applied) return;
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(5) as SchemaVersion | undefined;
if (applied) return;
// Check if column exists
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasWorkerPort = tableInfo.some(col => col.name === 'worker_port');
// Check if column exists
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasWorkerPort = tableInfo.some(col => col.name === 'worker_port');
if (!hasWorkerPort) {
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER');
console.log('[SessionStore] Added worker_port column to sdk_sessions table');
}
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(5, new Date().toISOString());
} catch (error: any) {
console.error('[SessionStore] Migration error:', error.message);
if (!hasWorkerPort) {
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER');
console.log('[SessionStore] Added worker_port column to sdk_sessions table');
}
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(5, new Date().toISOString());
}
/**
* Ensure prompt tracking columns exist (migration 6)
*/
private ensurePromptTrackingColumns(): void {
try {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(6) as SchemaVersion | undefined;
if (applied) return;
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(6) as SchemaVersion | undefined;
if (applied) return;
// Check sdk_sessions for prompt_counter
const sessionsInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasPromptCounter = sessionsInfo.some(col => col.name === 'prompt_counter');
// Check sdk_sessions for prompt_counter
const sessionsInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasPromptCounter = sessionsInfo.some(col => col.name === 'prompt_counter');
if (!hasPromptCounter) {
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0');
console.log('[SessionStore] Added prompt_counter column to sdk_sessions table');
}
// Check observations for prompt_number
const observationsInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
const obsHasPromptNumber = observationsInfo.some(col => col.name === 'prompt_number');
if (!obsHasPromptNumber) {
this.db.run('ALTER TABLE observations ADD COLUMN prompt_number INTEGER');
console.log('[SessionStore] Added prompt_number column to observations table');
}
// Check session_summaries for prompt_number
const summariesInfo = this.db.query('PRAGMA table_info(session_summaries)').all() as TableColumnInfo[];
const sumHasPromptNumber = summariesInfo.some(col => col.name === 'prompt_number');
if (!sumHasPromptNumber) {
this.db.run('ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER');
console.log('[SessionStore] Added prompt_number column to session_summaries table');
}
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(6, new Date().toISOString());
} catch (error: any) {
console.error('[SessionStore] Prompt tracking migration error:', error.message);
if (!hasPromptCounter) {
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0');
console.log('[SessionStore] Added prompt_counter column to sdk_sessions table');
}
// Check observations for prompt_number
const observationsInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
const obsHasPromptNumber = observationsInfo.some(col => col.name === 'prompt_number');
if (!obsHasPromptNumber) {
this.db.run('ALTER TABLE observations ADD COLUMN prompt_number INTEGER');
console.log('[SessionStore] Added prompt_number column to observations table');
}
// Check session_summaries for prompt_number
const summariesInfo = this.db.query('PRAGMA table_info(session_summaries)').all() as TableColumnInfo[];
const sumHasPromptNumber = summariesInfo.some(col => col.name === 'prompt_number');
if (!sumHasPromptNumber) {
this.db.run('ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER');
console.log('[SessionStore] Added prompt_number column to session_summaries table');
}
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(6, new Date().toISOString());
}
/**
* Remove UNIQUE constraint from session_summaries.sdk_session_id (migration 7)
*/
private removeSessionSummariesUniqueConstraint(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(7) as SchemaVersion | undefined;
if (applied) return;
// Check if UNIQUE constraint exists
const summariesIndexes = this.db.query('PRAGMA index_list(session_summaries)').all() as IndexInfo[];
const hasUniqueConstraint = summariesIndexes.some(idx => idx.unique === 1);
if (!hasUniqueConstraint) {
// Already migrated (no constraint exists)
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(7, new Date().toISOString());
return;
}
console.log('[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id...');
// Begin transaction
this.db.run('BEGIN TRANSACTION');
try {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(7) as SchemaVersion | undefined;
if (applied) return;
// Create new table without UNIQUE constraint
this.db.run(`
CREATE TABLE session_summaries_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sdk_session_id TEXT NOT NULL,
project TEXT NOT NULL,
request TEXT,
investigated TEXT,
learned TEXT,
completed TEXT,
next_steps TEXT,
files_read TEXT,
files_edited TEXT,
notes TEXT,
prompt_number INTEGER,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(sdk_session_id) REFERENCES sdk_sessions(sdk_session_id) ON DELETE CASCADE
)
`);
// Check if UNIQUE constraint exists
const summariesIndexes = this.db.query('PRAGMA index_list(session_summaries)').all() as IndexInfo[];
const hasUniqueConstraint = summariesIndexes.some(idx => idx.unique === 1);
// Copy data from old table
this.db.run(`
INSERT INTO session_summaries_new
SELECT id, sdk_session_id, project, request, investigated, learned,
completed, next_steps, files_read, files_edited, notes,
prompt_number, created_at, created_at_epoch
FROM session_summaries
`);
if (!hasUniqueConstraint) {
// Already migrated (no constraint exists)
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(7, new Date().toISOString());
return;
}
// Drop old table
this.db.run('DROP TABLE session_summaries');
console.log('[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id...');
// Rename new table
this.db.run('ALTER TABLE session_summaries_new RENAME TO session_summaries');
// Begin transaction
this.db.run('BEGIN TRANSACTION');
// Recreate indexes
this.db.run(`
CREATE INDEX idx_session_summaries_sdk_session ON session_summaries(sdk_session_id);
CREATE INDEX idx_session_summaries_project ON session_summaries(project);
CREATE INDEX idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`);
try {
// Create new table without UNIQUE constraint
this.db.run(`
CREATE TABLE session_summaries_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sdk_session_id TEXT NOT NULL,
project TEXT NOT NULL,
request TEXT,
investigated TEXT,
learned TEXT,
completed TEXT,
next_steps TEXT,
files_read TEXT,
files_edited TEXT,
notes TEXT,
prompt_number INTEGER,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(sdk_session_id) REFERENCES sdk_sessions(sdk_session_id) ON DELETE CASCADE
)
`);
// Commit transaction
this.db.run('COMMIT');
// Copy data from old table
this.db.run(`
INSERT INTO session_summaries_new
SELECT id, sdk_session_id, project, request, investigated, learned,
completed, next_steps, files_read, files_edited, notes,
prompt_number, created_at, created_at_epoch
FROM session_summaries
`);
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(7, new Date().toISOString());
// Drop old table
this.db.run('DROP TABLE session_summaries');
// Rename new table
this.db.run('ALTER TABLE session_summaries_new RENAME TO session_summaries');
// Recreate indexes
this.db.run(`
CREATE INDEX idx_session_summaries_sdk_session ON session_summaries(sdk_session_id);
CREATE INDEX idx_session_summaries_project ON session_summaries(project);
CREATE INDEX idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`);
// Commit transaction
this.db.run('COMMIT');
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(7, new Date().toISOString());
console.log('[SessionStore] Successfully removed UNIQUE constraint from session_summaries.sdk_session_id');
} catch (error: any) {
// Rollback on error
this.db.run('ROLLBACK');
throw error;
}
console.log('[SessionStore] Successfully removed UNIQUE constraint from session_summaries.sdk_session_id');
} catch (error: any) {
console.error('[SessionStore] Migration error (remove UNIQUE constraint):', error.message);
// Rollback on error
this.db.run('ROLLBACK');
throw error;
}
}
@@ -297,41 +285,37 @@ export class SessionStore {
* Add hierarchical fields to observations table (migration 8)
*/
private addObservationHierarchicalFields(): void {
try {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(8) as SchemaVersion | undefined;
if (applied) return;
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(8) as SchemaVersion | undefined;
if (applied) return;
// Check if new fields already exist
const tableInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
const hasTitle = tableInfo.some(col => col.name === 'title');
// Check if new fields already exist
const tableInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
const hasTitle = tableInfo.some(col => col.name === 'title');
if (hasTitle) {
// Already migrated
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(8, new Date().toISOString());
return;
}
console.log('[SessionStore] Adding hierarchical fields to observations table...');
// Add new columns
this.db.run(`
ALTER TABLE observations ADD COLUMN title TEXT;
ALTER TABLE observations ADD COLUMN subtitle TEXT;
ALTER TABLE observations ADD COLUMN facts TEXT;
ALTER TABLE observations ADD COLUMN narrative TEXT;
ALTER TABLE observations ADD COLUMN concepts TEXT;
ALTER TABLE observations ADD COLUMN files_read TEXT;
ALTER TABLE observations ADD COLUMN files_modified TEXT;
`);
// Record migration
if (hasTitle) {
// Already migrated
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(8, new Date().toISOString());
console.log('[SessionStore] Successfully added hierarchical fields to observations table');
} catch (error: any) {
console.error('[SessionStore] Migration error (add hierarchical fields):', error.message);
return;
}
console.log('[SessionStore] Adding hierarchical fields to observations table...');
// Add new columns
this.db.run(`
ALTER TABLE observations ADD COLUMN title TEXT;
ALTER TABLE observations ADD COLUMN subtitle TEXT;
ALTER TABLE observations ADD COLUMN facts TEXT;
ALTER TABLE observations ADD COLUMN narrative TEXT;
ALTER TABLE observations ADD COLUMN concepts TEXT;
ALTER TABLE observations ADD COLUMN files_read TEXT;
ALTER TABLE observations ADD COLUMN files_modified TEXT;
`);
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(8, new Date().toISOString());
console.log('[SessionStore] Successfully added hierarchical fields to observations table');
}
/**
@@ -339,86 +323,82 @@ export class SessionStore {
* The text field is deprecated in favor of structured fields (title, subtitle, narrative, etc.)
*/
private makeObservationsTextNullable(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(9) as SchemaVersion | undefined;
if (applied) return;
// Check if text column is already nullable
const tableInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
const textColumn = tableInfo.find(col => col.name === 'text');
if (!textColumn || textColumn.notnull === 0) {
// Already migrated or text column doesn't exist
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(9, new Date().toISOString());
return;
}
console.log('[SessionStore] Making observations.text nullable...');
// Begin transaction
this.db.run('BEGIN TRANSACTION');
try {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(9) as SchemaVersion | undefined;
if (applied) return;
// Create new table with text as nullable
this.db.run(`
CREATE TABLE observations_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sdk_session_id TEXT NOT NULL,
project TEXT NOT NULL,
text TEXT,
type TEXT NOT NULL CHECK(type IN ('decision', 'bugfix', 'feature', 'refactor', 'discovery', 'change')),
title TEXT,
subtitle TEXT,
facts TEXT,
narrative TEXT,
concepts TEXT,
files_read TEXT,
files_modified TEXT,
prompt_number INTEGER,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(sdk_session_id) REFERENCES sdk_sessions(sdk_session_id) ON DELETE CASCADE
)
`);
// Check if text column is already nullable
const tableInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
const textColumn = tableInfo.find(col => col.name === 'text');
// Copy data from old table (all existing columns)
this.db.run(`
INSERT INTO observations_new
SELECT id, sdk_session_id, project, text, type, title, subtitle, facts,
narrative, concepts, files_read, files_modified, prompt_number,
created_at, created_at_epoch
FROM observations
`);
if (!textColumn || textColumn.notnull === 0) {
// Already migrated or text column doesn't exist
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(9, new Date().toISOString());
return;
}
// Drop old table
this.db.run('DROP TABLE observations');
console.log('[SessionStore] Making observations.text nullable...');
// Rename new table
this.db.run('ALTER TABLE observations_new RENAME TO observations');
// Begin transaction
this.db.run('BEGIN TRANSACTION');
// Recreate indexes
this.db.run(`
CREATE INDEX idx_observations_sdk_session ON observations(sdk_session_id);
CREATE INDEX idx_observations_project ON observations(project);
CREATE INDEX idx_observations_type ON observations(type);
CREATE INDEX idx_observations_created ON observations(created_at_epoch DESC);
`);
try {
// Create new table with text as nullable
this.db.run(`
CREATE TABLE observations_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sdk_session_id TEXT NOT NULL,
project TEXT NOT NULL,
text TEXT,
type TEXT NOT NULL CHECK(type IN ('decision', 'bugfix', 'feature', 'refactor', 'discovery', 'change')),
title TEXT,
subtitle TEXT,
facts TEXT,
narrative TEXT,
concepts TEXT,
files_read TEXT,
files_modified TEXT,
prompt_number INTEGER,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(sdk_session_id) REFERENCES sdk_sessions(sdk_session_id) ON DELETE CASCADE
)
`);
// Commit transaction
this.db.run('COMMIT');
// Copy data from old table (all existing columns)
this.db.run(`
INSERT INTO observations_new
SELECT id, sdk_session_id, project, text, type, title, subtitle, facts,
narrative, concepts, files_read, files_modified, prompt_number,
created_at, created_at_epoch
FROM observations
`);
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(9, new Date().toISOString());
// Drop old table
this.db.run('DROP TABLE observations');
// Rename new table
this.db.run('ALTER TABLE observations_new RENAME TO observations');
// Recreate indexes
this.db.run(`
CREATE INDEX idx_observations_sdk_session ON observations(sdk_session_id);
CREATE INDEX idx_observations_project ON observations(project);
CREATE INDEX idx_observations_type ON observations(type);
CREATE INDEX idx_observations_created ON observations(created_at_epoch DESC);
`);
// Commit transaction
this.db.run('COMMIT');
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(9, new Date().toISOString());
console.log('[SessionStore] Successfully made observations.text nullable');
} catch (error: any) {
// Rollback on error
this.db.run('ROLLBACK');
throw error;
}
console.log('[SessionStore] Successfully made observations.text nullable');
} catch (error: any) {
console.error('[SessionStore] Migration error (make text nullable):', error.message);
// Rollback on error
this.db.run('ROLLBACK');
throw error;
}
}
@@ -426,86 +406,82 @@ export class SessionStore {
* Create user_prompts table with FTS5 support (migration 10)
*/
private createUserPromptsTable(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(10) as SchemaVersion | undefined;
if (applied) return;
// Check if table already exists
const tableInfo = this.db.query('PRAGMA table_info(user_prompts)').all() as TableColumnInfo[];
if (tableInfo.length > 0) {
// Already migrated
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
return;
}
console.log('[SessionStore] Creating user_prompts table with FTS5 support...');
// Begin transaction
this.db.run('BEGIN TRANSACTION');
try {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(10) as SchemaVersion | undefined;
if (applied) return;
// Create main table (using claude_session_id since sdk_session_id is set asynchronously by worker)
this.db.run(`
CREATE TABLE user_prompts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
claude_session_id TEXT NOT NULL,
prompt_number INTEGER NOT NULL,
prompt_text TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(claude_session_id) REFERENCES sdk_sessions(claude_session_id) ON DELETE CASCADE
);
// Check if table already exists
const tableInfo = this.db.query('PRAGMA table_info(user_prompts)').all() as TableColumnInfo[];
if (tableInfo.length > 0) {
// Already migrated
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
return;
}
CREATE INDEX idx_user_prompts_claude_session ON user_prompts(claude_session_id);
CREATE INDEX idx_user_prompts_created ON user_prompts(created_at_epoch DESC);
CREATE INDEX idx_user_prompts_prompt_number ON user_prompts(prompt_number);
CREATE INDEX idx_user_prompts_lookup ON user_prompts(claude_session_id, prompt_number);
`);
console.log('[SessionStore] Creating user_prompts table with FTS5 support...');
// Create FTS5 virtual table
this.db.run(`
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
prompt_text,
content='user_prompts',
content_rowid='id'
);
`);
// Begin transaction
this.db.run('BEGIN TRANSACTION');
// Create triggers to sync FTS5
this.db.run(`
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
try {
// Create main table (using claude_session_id since sdk_session_id is set asynchronously by worker)
this.db.run(`
CREATE TABLE user_prompts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
claude_session_id TEXT NOT NULL,
prompt_number INTEGER NOT NULL,
prompt_text TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(claude_session_id) REFERENCES sdk_sessions(claude_session_id) ON DELETE CASCADE
);
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
END;
CREATE INDEX idx_user_prompts_claude_session ON user_prompts(claude_session_id);
CREATE INDEX idx_user_prompts_created ON user_prompts(created_at_epoch DESC);
CREATE INDEX idx_user_prompts_prompt_number ON user_prompts(prompt_number);
CREATE INDEX idx_user_prompts_lookup ON user_prompts(claude_session_id, prompt_number);
`);
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`);
// Create FTS5 virtual table
this.db.run(`
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
prompt_text,
content='user_prompts',
content_rowid='id'
);
`);
// Commit transaction
this.db.run('COMMIT');
// Create triggers to sync FTS5
this.db.run(`
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
END;
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`);
// Commit transaction
this.db.run('COMMIT');
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
console.log('[SessionStore] Successfully created user_prompts table with FTS5 support');
} catch (error: any) {
// Rollback on error
this.db.run('ROLLBACK');
throw error;
}
console.log('[SessionStore] Successfully created user_prompts table with FTS5 support');
} catch (error: any) {
console.error('[SessionStore] Migration error (create user_prompts table):', error.message);
// Rollback on error
this.db.run('ROLLBACK');
throw error;
}
}
@@ -990,25 +966,17 @@ export class SessionStore {
for (const row of rows) {
// Parse files_read
if (row.files_read) {
try {
const files = JSON.parse(row.files_read);
if (Array.isArray(files)) {
files.forEach(f => filesReadSet.add(f));
}
} catch {
// Skip invalid JSON
const files = JSON.parse(row.files_read);
if (Array.isArray(files)) {
files.forEach(f => filesReadSet.add(f));
}
}
// Parse files_modified
if (row.files_modified) {
try {
const files = JSON.parse(row.files_modified);
if (Array.isArray(files)) {
files.forEach(f => filesModifiedSet.add(f));
}
} catch {
// Skip invalid JSON
const files = JSON.parse(row.files_modified);
if (Array.isArray(files)) {
files.forEach(f => filesModifiedSet.add(f));
}
}
}
+1
View File
@@ -495,6 +495,7 @@ export const migration007: Migration = {
}
};
/**
* All migrations in order
*/