feat: Add ROI tracking with discovery_tokens for observations and session summaries (#111)

* feat: Add discovery_tokens for ROI tracking in observations and session summaries

- Introduced `discovery_tokens` column in `observations` and `session_summaries` tables to track token costs associated with discovering and creating each observation and summary.
- Updated relevant services and hooks to calculate and display ROI metrics based on discovery tokens.
- Enhanced context economics reporting to include savings from reusing previous observations.
- Implemented migration to ensure the new column is added to existing tables.
- Adjusted data models and sync processes to accommodate the new `discovery_tokens` field.

* refactor: streamline context hook by removing unused functions and updating terminology

- Removed the estimateTokens and getObservations helper functions as they were not utilized.
- Updated the legend and output messages to replace "discovery" with "work" for clarity.
- Changed the emoji representation for different observation types to better reflect their purpose.
- Enhanced output formatting for improved readability and understanding of token usage.

* Refactor user-message-hook and context-hook for improved clarity and functionality

- Updated user-message-hook.js to enhance error messaging and improve variable naming for clarity.
- Modified context-hook.ts to include a new column key section, improved context index instructions, and added emoji icons for observation types.
- Adjusted footer messages in context-hook.ts to emphasize token savings and access to past research.
- Changed user-message-hook.ts to update the feedback and support message for clarity.

* fix: Critical ROI tracking fixes from PR review

Addresses critical findings from PR #111 review:

1. **Fixed incorrect discovery token calculation** (src/services/worker/SDKAgent.ts)
   - Changed from passing cumulative total to per-response delta
   - Now correctly tracks token cost for each observation/summary
   - Captures token state before/after response processing
   - Prevents all observations getting inflated cumulative values

2. **Fixed schema version mismatch** (src/services/sqlite/SessionStore.ts)
   - Changed ensureDiscoveryTokensColumn() from version 11 to version 7
   - Now matches migration007 definition in migrations.ts
   - Ensures consistent version tracking across migration system

These fixes ensure ROI metrics accurately reflect token costs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

src/services/sync/ChromaSync.ts

@@ -36,6 +36,7 @@ interface StoredObservation {
   files_read: string | null; // JSON
   files_modified: string | null; // JSON
   prompt_number: number;
+  discovery_tokens: number; // ROI metrics
   created_at: string;
   created_at_epoch: number;
 }
@@ -51,6 +52,7 @@ interface StoredSummary {
   next_steps: string | null;
   notes: string | null;
   prompt_number: number;
+  discovery_tokens: number; // ROI metrics
   created_at: string;
   created_at_epoch: number;
 }
@@ -345,7 +347,8 @@ export class ChromaSync {
     project: string,
     obs: ParsedObservation,
     promptNumber: number,
-    createdAtEpoch: number
+    createdAtEpoch: number,
+    discoveryTokens: number = 0
   ): Promise<void> {
     // Convert ParsedObservation to StoredObservation format
     const stored: StoredObservation = {
@@ -362,6 +365,7 @@ export class ChromaSync {
       files_read: JSON.stringify(obs.files_read),
       files_modified: JSON.stringify(obs.files_modified),
       prompt_number: promptNumber,
+      discovery_tokens: discoveryTokens,
       created_at: new Date(createdAtEpoch * 1000).toISOString(),
       created_at_epoch: createdAtEpoch
     };
@@ -387,7 +391,8 @@ export class ChromaSync {
     project: string,
     summary: ParsedSummary,
     promptNumber: number,
-    createdAtEpoch: number
+    createdAtEpoch: number,
+    discoveryTokens: number = 0
   ): Promise<void> {
     // Convert ParsedSummary to StoredSummary format
     const stored: StoredSummary = {
@@ -401,6 +406,7 @@ export class ChromaSync {
       next_steps: summary.next_steps,
       notes: summary.notes,
       prompt_number: promptNumber,
+      discovery_tokens: discoveryTokens,
       created_at: new Date(createdAtEpoch * 1000).toISOString(),
       created_at_epoch: createdAtEpoch
     };