UX redesign: installer + provider rename + /learn-codebase + welcome card + SessionStart hint (#2255)
* feat(ux): claude-mem UX improvements with installer enhancements
Squashed PR #2156 commits for clean rebase onto main:
- feat(installer): add provider selection, model prompt, worker auto-start
- refactor: rename *Agent provider classes to *Provider
- feat: add /learn-codebase skill and viewer welcome card
- feat(worker): inject welcome hint when project has zero observations
- fix(pr-2156): address greptile review comments
- fix(pr-2156): address coderabbit review comments
- fix(pr-2156): persist CLAUDE_MEM_PROVIDER for non-claude in non-TTY mode
- fix(pr-2156): file-backed settings reads in installer + env-first SKILL doc
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* build: rebuild plugin artifacts after rebase onto v12.4.7
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(skills): strip claude-mem internals from learn-codebase
The learn-codebase skill, install next-step copy, WelcomeCard, and
welcome-hint previously walked the primary agent through worker endpoints
and synthetic observation payloads. The PostToolUse hook already captures
every Read/Edit the agent makes — the agent should have no awareness that
the memory layer exists. Collapse the skill to one instruction ("read every
source file in full") and rephrase touchpoints to describe only what the
user observes (Claude reading files), not what happens behind the scenes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(sync): preflight version mismatch + settings-aware port resolution
Two related fixes for build-and-sync's worker restart step:
1. Read CLAUDE_MEM_WORKER_PORT from ~/.claude-mem/settings.json the same
way the worker does, instead of computing the default port from the
uid alone. Previously, users with a custom port saw a misleading
"Worker not running" message because the restart POST hit the wrong
port and got ECONNREFUSED.
2. Add a preflight check that aborts the sync when the running worker's
reported version does not match the version we are about to build.
Claude Code's plugin loader pins the worker to a specific cache
version per session, so syncing into a newer cache directory has no
effect until the user runs `claude plugin update thedotmack/claude-mem`
to bump the pin. The preflight surfaces this explicitly with the exact
command to run; --force bypasses it for intentional cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(learn-codebase): note sed for partial reads of large files
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: strip comments codebase-wide
Removed prose comments from all tracked source. Preserved directives
(@ts-ignore, eslint-disable, biome-ignore, prettier-ignore, triple-slash
references, webpack magic, shebangs). Deleted two tests that asserted
on comment text rather than runtime behavior.
Net: 401 files, -14,587 / +389 lines, -10.4% bytes.
Verified: typecheck passes, build passes, test count unchanged from
baseline (22 pre-existing fails, all unrelated).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(installer): move runtime setup into npx, eliminate hook dead air
Smart-install ran 3 times during a fresh install — the worst run was silent,
fired by Claude Code's Setup hook after `claude plugin install`, producing
~30s of dead air that looked like the plugin was hung.
This change makes `npx claude-mem install` the single place heavy work
happens, with a visible spinner. Hooks become runtime-only.
- New `src/npx-cli/install/setup-runtime.ts` module: ensureBun, ensureUv,
installPluginDependencies, read/writeInstallMarker, isInstallCurrent.
Marker schema preserved exactly ({version, bun, uv, installedAt}) so
ContextBuilder and BranchManager readers keep working.
- `npx claude-mem install`: ungated copy/register/enable for every IDE,
inserts a "Setting up runtime" task with honest "first install can take
~30s" spinner. The claude-code shell-out to `claude plugin install` is
removed — npx already populated everything Claude reads.
- New `npx claude-mem repair` command for post-`claude plugin update`
recovery, force-reinstalls runtime.
- Setup hook now runs `plugin/scripts/version-check.js` (29ms wall) instead
of smart-install. Mismatch prints "run: npx claude-mem repair" on stderr.
Always exits 0 (non-blocking, per CLAUDE.md exit-code strategy).
- SessionStart loses the smart-install entry; 2 hooks remain (worker start,
context fetch).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(installer): delete smart-install sources, retarget tests
- Delete scripts/smart-install.js + plugin/scripts/smart-install.js (both
are source files kept in sync manually; both must go).
- Delete tests/smart-install.test.ts (covered surface is gone).
- tests/plugin-scripts-line-endings: drop smart-install.js entry.
- tests/infrastructure/plugin-distribution: retarget two assertions at
version-check.js (the new Setup hook script).
- New tests/setup-runtime.test.ts: 9 tests covering marker read/write,
isInstallCurrent semantics. Marker schema invariant verified.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(installer): describe npx-driven setup + version-check Setup hook
Sweep public docs and architecture notes to reflect the new flow:
npx installer does Bun/uv setup with a visible spinner; Setup hook runs
sub-100ms version-check.js; users hit `npx claude-mem repair` after a
`claude plugin update`.
- docs/architecture-overview.md: hook lifecycle table + npx flow paragraph
- docs/public/configuration.mdx: tree + hook config example
- docs/public/development.mdx: build output line
- docs/public/hooks-architecture.mdx: full rewrite of pre-hook section,
timing table, performance table
- docs/public/architecture/{overview,hooks,worker-service}.mdx: tree
comments, JSON config example, Bun requirement section
docs/reports/* untouched (historical incident reports).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(install): mergeSettings writes via USER_SETTINGS_PATH
Greptile P1 (#2156): `settingsFilePath()` only resolved
`process.env.CLAUDE_MEM_DATA_DIR`, while `getSetting()` reads via
`USER_SETTINGS_PATH` which `resolveDataDir()` populates from BOTH the env
var AND a `CLAUDE_MEM_DATA_DIR` entry persisted in
`~/.claude-mem/settings.json`. Result: a user with the data dir saved in
settings.json but not exported in their shell would have provider/model
settings silently written to `~/.claude-mem/settings.json` while
`getSetting()` read from `/custom/path/settings.json` — read/write split.
Drop `settingsFilePath()` and the now-unused `homedir` import; reuse the
already-imported `USER_SETTINGS_PATH` constant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli): parse --provider, --model, --no-auto-start install flags
Greptile P1 (#2156): InstallOptions has fields `provider`, `model`,
`noAutoStart`, but the install case in the npx-cli switch only parsed
`--ide`. The other three flags were silently dropped — `npx claude-mem
install --provider gemini` was a no-op.
Extract a `parseInstallOptions(argv)` helper, share it between the bare
`npx claude-mem` and `npx claude-mem install` paths, and validate
`--provider` against the allowed set. Update help text accordingly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(install): pipe runtime-setup output, always show IDE multiselect
Two issues caught in a docker test of the installer:
1. The bun.sh installer, uv installer, and `bun install` were using
stdio: 'inherit', dumping their stdout/stderr through clack's spinner
region — visible as raw "downloading uv 0.11.8…" / "Checked 58
installs across 38 packages…" text streaming under the spinner. Switch
to stdio: 'pipe' and surface captured stderr only on failure (via a
shared describeExecError() helper that includes stdout when stderr is
empty). Spinner stays clean on the happy path.
2. promptForIDESelection() silently picked claude-code when no IDEs were
detected, never showing the user the multiselect. On a fresh machine
with no IDEs present yet (e.g. our docker test container), the user
never got to choose. Now: always show the full IDE list when
interactive; mark detected ones with [detected] hints and pre-select
them; show a warn line if zero are detected explaining they should pick
what they plan to use. Non-TTY callers still get the silent
claude-code default at the call site (unchanged).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(install): skip marketplace work for claude-code-only, offer to install Claude Code
Two related UX fixes from a docker test:
**Delay between "Saved Claude model=…" and "Plugin files copied OK"**
After dropping the needsManualInstall gate, every install was unconditionally
running `copyPluginToMarketplace` (which copied the entire root node_modules
tree — thousands of files, dozens of seconds) and `runNpmInstallInMarketplace`
(npm install --production) even when only claude-code was selected. Neither
is needed for claude-code: that path uses the plugin cache dir + the
installed_plugins.json + enabledPlugins flag, all of which we already write.
- Drop `node_modules` from `copyPluginToMarketplace`'s allowed-entries list;
the dependency-install task populates it on the destination side anyway.
- Re-introduce `needsMarketplace = selectedIDEs.some(id => id !== 'claude-code')`
scoped *only* to `copyPluginToMarketplace`, `runNpmInstallInMarketplace`,
and the pre-install `shutdownWorkerAndWait` (also pointless for claude-code-
only flows since we're not overwriting the worker's running cache dir
source). All other tasks (cache copy, register, enable, runtime setup) stay
unconditional.
**Claude Code missing → silent install of an IDE that isn't there**
When the user picked claude-code on a machine without it (e.g. a fresh
container), the install completed but `claude` was unavailable and the only
hint was a generic warn line. Replace with an explicit pre-flight prompt:
Claude Code is not installed. Claude-mem works best in Claude Code, but
also works with the IDEs below.
? Install Claude Code now?
◆ Yes — install Claude Code (recommended)
◯ No — pick another IDE below
◯ Cancel installation
If the user picks "Yes", run `curl -fsSL https://claude.ai/install.sh | bash`
(or the PowerShell equivalent on Windows), then re-detect IDEs and proceed
with claude-code pre-selected. If the install fails or the user picks "No",
the multiselect still appears with claude-code visible (just unmarked
[detected]), so they can opt in or pick another IDE.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(install): detect Claude Code via `claude` CLI, not ~/.claude dir
The directory `~/.claude` can exist (e.g. mounted in Docker, or created
by tooling) without Claude Code actually being installed. Detect the
`claude` command in PATH instead so the installer correctly offers to
install Claude Code when missing.
* docs(learn-codebase): add reviewer note explaining the cost tradeoff
The skill intentionally reads every file in full to build a cognitive
cache that pays off across the rest of the project. Add a brief note
so reviewers (human or bot) understand the tradeoff before flagging
the unbounded read as a cost issue.
* fix: address Greptile P1 feedback on welcome hint and learn-codebase
- SearchRoutes: skip welcome hint when caller passes ?full=true so
explicit full-context requests aren't intercepted by the hint.
- learn-codebase: replace `sed` instruction with the Read tool's
offset/limit parameters, since Bash is gated in Claude Code by
default.
* feat(install): ASCII-animated logo splash on interactive install
Plays a ~1s bloom animation of the claude-mem sunburst logomark when
the installer starts in an interactive terminal — geometrically rendered
via 12 ray curves around a center disc, in the brand orange. The
wordmark and tagline type on alongside the final frame.
Auto-skipped on non-TTY, in CI, when NO_COLOR or CLAUDE_MEM_NO_BANNER
is set, or when the terminal is too narrow.
Inspired by ghostty +boo.
* feat(banner): replace rotation frames with angular-sector bloom generator
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(banner): replace rotation frames with angular-sector bloom generator
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(banner): three-act choreography renderer with radial gradient and diff redraw
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(banner): update preview script to support small/medium/hero tier selection
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(docker): add COLORTERM=truecolor to test-installer sandbox
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(install): auto-apply PATH for Claude Code with spinner UX
The Claude Code install.sh prints a Setup notes block telling users to
manually edit "your shell config file" to add ~/.local/bin to PATH —
which left fresh installs unable to launch claude from the command line.
After a successful install, detect ~/.local/bin/claude on disk and, if
the dir is missing from PATH, append the right export line to .zshrc /
.bash_profile / .bashrc / fish config (idempotent, marked with a
comment). Also updates process.env.PATH for the current install run.
Wraps the curl|bash install in a clack spinner (interactive only) so the
~4 minute native-build download doesn't look frozen — output is captured
silently and dumped on failure for debuggability. Non-interactive mode
keeps inherited stdio for CI logs.
Verified end-to-end in the test-installer docker sandbox: spinner
animates, .bashrc gets the export, fresh login shell resolves claude.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(banner): video-frame ASCII renderer with three-act choreography
Generator switched from a single Jimp-rendered logo to pre-extracted
video frames concatenated with \x01 separators and gzip-deflated, ported
from ghostty's boo wire format. Renderer rewritten around three acts
(ignite → stagger bloom → text reveal + breathe) with adaptive sizing,
radial gradient, and diff-based redraw.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(onboarding): unify install / SessionStart / viewer around one first-success moment
Three surfaces now point at the same north-star moment — open the viewer, do
anything in Claude Code, watch an observation appear within seconds — with the
same verbatim timing and privacy lines, and a single canonical "how it works"
explainer instead of three diverging copies.
- Canonical explainer at src/services/worker/onboarding-explainer.md served via
GET /api/onboarding/explainer; mirrored into plugin/skills/how-it-works/SKILL.md
- SessionStart welcome hint rewritten as third-person status (no imperatives
Claude tries to execute), pinned with a default-value regression test
- Post-install Next Steps reframed as "two paths": passive default + optional
/learn-codebase front-load; drops /mem-search and /knowledge-agent from this
surface; adds verbatim timing + privacy lines and /how-it-works link
- /api/stats response gains firstObservationAt for the viewer stat row
- Viewer WelcomeCard branches on observationCount === 0: empty state shows live
worker-connection dot + "waiting for activity"; has-data state shows
observations · projects · since [date] and two example prompts. v2 dismiss key
- jimp added to package.json to fix pre-existing banner-frame build break
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(banner): play unconditionally; only honor CLAUDE_MEM_NO_BANNER
The 128-col / TTY / CI / NO_COLOR gates silently swallowed the banner in
narrower terminals, CI logs, and any non-TTY pipe — including Docker runs
where -it should preserve the experience but column width was the wrong
gate. Remove the implicit gates; keep the explicit opt-out only.
If a frame wraps in a narrow terminal, that's better than the banner
not playing at all.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* revert(banner): restore 15:33 gating logic per user request
Reverts eb6fc157. Restores isBannerEnabled to the state at commit
8e448015 (2026-04-30 15:33): TTY check, !CI, !NO_COLOR, !CLAUDE_MEM_NO_BANNER,
and cols >= BANNER.width.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(install): wrap remaining slow steps with spinners
Each IDE installer (Cursor, Gemini CLI, OpenCode, Windsurf, OpenClaw,
Codex CLI, MCP integrations) now runs inside a clack task spinner with
per-step progress messages instead of silent dynamic-import + cpSync.
Pre-overwrite worker shutdown (up to 10s) and the post-install health
probe (up to 3s) also get spinners.
Internal console.log/error/warn from each IDE installer is buffered
during the spinner; if the install fails, captured output is replayed
afterward via log.warn so users can see what broke.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(review): observation count + IDE pre-selection regressions
WelcomeCard's "no observations yet" empty state was triggered when a
project filter narrowed the feed to zero rows, even with thousands of
observations elsewhere. Source the count from global stats.database
to match firstObservationAt's scope.
Restore initialValues: [] in the IDE multiselect — pre-selecting every
detected IDE was the exact regression #2106 was filed for.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(install): trichotomy worker state + cache fallback for script path
ensureWorkerStarted now returns 'ready' | 'warming' | 'dead' instead of
boolean. The spawned-but-still-warming case (common in Docker cold
starts and slow first-time inits) was being misreported as 'did not
start', which contradicted the next-steps panel saying 'still starting
up'. Install task message and Next Steps headline now agree on the
actual state.
Also fixes the actual root cause of 'Worker did not start' on
claude-code-only installs: the worker script path was hardcoded to the
marketplace dir, which is left empty when no non-claude-code IDE is
selected. Now falls back to pluginCacheDirectory(version) when the
marketplace copy isn't present.
Verified end-to-end in docker/claude-mem with --ide claude-code,
--ide cursor, and a fresh container — install task and headline
agree on 'Worker ready at http://localhost:<port>' in all cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: align CLAUDE.md and public docs with current code
Sweep across CLAUDE.md and 10 high-traffic docs/public/ MDX files to
remove point-in-time references and align with the actual current
shape of the codebase. Highlights:
- Hardcoded port 37777 → per-user formula (37700 + uid % 100) on the
front-door pages (introduction, installation, configuration,
architecture/overview, architecture/worker-service, troubleshooting,
hooks-architecture, platform-integration).
- Default model 'sonnet' → 'claude-haiku-4-5-20251001' (matches
SettingsDefaultsManager).
- Node 18 → 20 (matches package.json engines).
- Lifecycle hook count corrected (5 events).
- Removed the nonexistent 'Smart Install' component and pre-built
directory tree referencing files that no longer exist
(context-hook.ts, save-hook.ts, cleanup-hook.ts, etc.); replaced
with the real worker dispatcher shape.
- Removed CLAUDE.md '#2101' issue tag (kept the design rationale).
- Replaced obsolete hooks.json example with a description of the real
bun-runner.js / worker-service.cjs hook event shape.
Lower-traffic doc pages still hardcode 37777 — left for a separate
global pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(scripts): land strip-comments around real parsers (postcss, remark, parse5)
Each language gets a real parser to locate comments, then we splice ranges
out of the original source. The library never serializes — that's how
remark-stringify produced 243 reformat-noise diffs in the first attempt
versus the 21 real strip targets here.
JS/TS/JSX -> ts.createSourceFile + getLeadingCommentRanges
CSS/SCSS -> postcss.parse + walkComments + node.source offsets
MD/MDX -> remark-parse (+ remark-mdx) + AST html / mdx-expression nodes
HTML -> parse5 with sourceCodeLocationInfo
shell/py -> kept hand-rolled hash stripper (no library worth the dep)
Preserves: shebangs, @ts-* directives, eslint-disable, biome-ignore,
prettier-ignore, triple-slash refs, webpack magic, /*! license keep,
@strip-comments-keep file marker. JS/TS handler runs a parse-roundtrip
check and refuses to write if syntax errors increased (catches the
worker-utils.ts breakage class from the 2026-04-29 attempt).
npm scripts:
strip-comments (apply)
strip-comments:check (CI-style, exits non-zero if changes needed)
strip-comments:dry-run (list, no writes)
Verified --check on this repo: 21 changes, -4.0% bytes, no parse-error
regressions, no reformat-suspect false positives.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: strip comments codebase-wide via parser-backed tool
21 files changed, -17,550 bytes (-4.0%) of narrative comments removed
across .ts / .tsx / .js / .mjs and the .gitignore. JS/TS comments stripped
via ts.createSourceFile + getLeadingCommentRanges — same canonical lexer,
same behavior as the 2026-04-29 strip, no reformat noise.
Preexisting baseline (unchanged):
typecheck: 16 errors at HEAD, 16 errors after strip (line numbers shift,
no new error classes — verified via diff of sorted error lists)
build: fails at HEAD with CrushHooksInstaller.js unresolved import
(preexisting, unrelated to this strip)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(install): drop Crush integration references after extract
The Crush integration was extracted to its own branch on May 1, but the
import at install.ts:280 (and the case block + ide-detection entry +
McpIntegrations config + npx-cli help text) still referenced the now-
removed CrushHooksInstaller.js, breaking the build.
Removes:
- case 'crush' block in install.ts
- crush entry in ide-detection.ts
- CRUSH_CONFIG and registration in McpIntegrations.ts
- 'crush' from the IDE Identifiers help line in index.ts
Rebuilds worker-service.cjs to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(banner): mark generated banner-frames.ts with @strip-comments-keep
Without this, every build/strip cycle ping-pongs five lines of doc
comments in and out of the auto-generated output. The keep-marker tells
strip-comments.ts to skip the file entirely.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(build): drop banner-frame regen from build script
generate-banner-frames.mjs requires PNG frames in /tmp/cmem-banner-frames
that only exist after the maintainer runs ffmpeg locally on the source
video. CI has neither the video nor the frames, so the build broke on
Windows. The output (src/npx-cli/banner-frames.ts) is committed, so the
regen is a one-shot dev step — not a build step. Run the script directly
when the video changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(worker): unstick the spinner — kill claim-self-lock, wake on fail, auto-broadcast
Three surgical changes that cure the stuck-spinner bug at the source.
Phase 1.1 (L9): claimNextMessage no longer self-excludes its own worker_pid.
A single UPDATE-RETURNING grabs the oldest pending row by id. Removes the
LiveWorkerPidsProvider plumbing that was never injected — Supervisor enforces
single-worker via PID file, so the multi-worker SQL was defending against a
configuration the project does not support.
Phase 1.2 (L19): SessionManager.markMessageFailed wraps PendingMessageStore.markFailed
and emits 'message' on the per-session EventEmitter. The iterator's waitForMessage
now wakes immediately on re-pend instead of parking for 3 minutes. ResponseProcessor
and SessionRoutes routed through the new wrapper.
Phase 1.3 (L24): PendingMessageStore takes an optional onMutate callback fired
from every mutator (enqueue, claimNextMessage, confirmProcessed, markFailed,
transitionMessagesTo, clearFailedOlderThan). SessionManager wires it; WorkerService
passes broadcastProcessingStatus. Ten manual broadcast calls deleted across
SessionCleanupHelper, SessionEventBroadcaster, SessionRoutes, DataRoutes, and
worker-service. Caller discipline becomes structurally impossible to forget.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(worker): delete dead code — legacy routes, processPendingQueues, decorative guards
Pure deletions. Phase 2 of kill-the-asshole-gates.
- Legacy /sessions/:sessionDbId/* routes (handleSessionInit, handleObservations,
handleSummarize, handleSessionStatus, handleSessionDelete, handleSessionComplete)
bypassed all five ingest gates and were a parallel write path. Folded the
initializeSession + broadcastNewPrompt + syncUserPrompt + ensureGeneratorRunning
+ broadcastSessionStarted work into the canonical /api/sessions/init handler so
the hook makes one round trip instead of two.
- processPendingQueues (~104 lines, zero callers) — replaced in Phase 6 by a
one-statement startup sweep.
- spawnInProgress Map and crashRecoveryScheduled Set — decorative dedupe over
generatorPromise and stillExists checks that already provide the real safety.
- STALE_GENERATOR_THRESHOLD_MS — pre-empted live generators and raced with the
finally block; the 3min idle timeout already kills zombies.
- MAX_SESSION_WALL_CLOCK_MS — ran a SELECT on every observation to enforce 24h.
Runaway-spend protection lives in the API key, not in claude-mem.
- Missing-id 400 in shared.ts ingestObservation — Zod already enforces min(1)
on contentSessionId and toolName at the route schema.
- SessionCompletionHandler import + completionHandler field on SessionRoutes
(orphaned after handler deletions).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(worker): SQL-backed getTotalQueueDepth — single source of truth
Was: iterate this.sessions.values() and sum getPendingCount per session.
Now: SELECT COUNT(*) FROM pending_messages WHERE status IN ('pending','processing').
The in-memory sessions Map drifted from the DB rows whenever a generator exited
without confirm/fail, leading to false-positive isProcessing in the UI. Phase 1.3's
auto-broadcast fires on every mutation, but it broadcast a stale Map count.
Reading from the DB makes the UI's spinner state match what the queue actually holds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(worker): typed abortReason replaces wasAborted boolean
Was: a boolean wasAborted that lumped every abort together. The finally block
branched on !wasAborted, so any abort skipped restart — including idle aborts
with pending work, which is exactly the case where we DO want to restart.
Now: ActiveSession.abortReason is a typed enum 'idle' | 'shutdown' | 'overflow'
| 'restart-guard'. The finally block consumes the reason and only skips restart
for 'shutdown' and 'restart-guard'. Idle and overflow aborts fall through, so
if pending work exists they trigger restart correctly.
Dropped 'stale' and 'wall-clock' from the union — Phase 2 deleted those paths.
Natural-completion abort (post-success) intentionally has no reason; it's not
gating restart logic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(worker): unify the two generator-exit finally blocks
Was: worker-service.ts:startSessionProcessor and SessionRoutes:ensureGeneratorRunning
each had their own ~70-line finally block with divergent restart-guard handling.
The worker-service path called terminateSession on RestartGuard trip and orphaned
pending rows (the L16 bug); the SessionRoutes path drained them. Two places to
update when rules changed.
Now: handleGeneratorExit in src/services/worker/session/GeneratorExitHandler.ts
owns the contract:
1. Always kill the SDK subprocess if alive.
2. Always drain processingMessageIds via sessionManager.markMessageFailed
(which wakes the iterator — Phase 1.2).
3. shutdown / restart-guard reasons: drain pending rows via
transitionMessagesTo('failed'), finalize, remove from Map. Fixes L16.
4. pendingCount=0: finalize normally and remove from Map.
5. pendingCount>0: backoff respawn via per-session respawnTimer (no global Set;
Phase 2.4 deleted that). RestartGuard trip drains to 'abandoned'.
Both finally blocks are now ~10-line wrappers that translate local state into the
canonical abortReason and delegate. Restored completionHandler injection into
SessionRoutes (was dropped in Phase 2 cleanup; needed by the unified helper for
finalizeSession).
Behavior change: SessionRoutes' previous "keep idle session in memory" was
deliberately replaced by the plan's "remove from Map on natural completion" —
next observation reinitializes via getMessageIterator → initializeSession.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(worker): startup orphan sweep — reset 'processing' rows at boot
When the worker dies (crash, kill, restart), any pending_messages rows it left
in 'processing' state are by definition orphans (the only worker is dead).
Single SQL UPDATE at boot resets them to 'pending' so the iterator can claim
them again. Replaces the deleted processPendingQueues function (Phase 2.2).
Runs in initializeBackground after dbManager.initialize() and before the
initializationComplete middleware releases blocked HTTP requests, so no
in-flight request can race the sweep. NOT on a periodic timer — after boot,
every 'processing' row has a live consumer and a periodic sweep would race.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(worker): simplify enqueue catch, replace memorySessionId throw with re-pend
7.1: queueObservation's catch was logging two ERROR-level messages and rethrowing.
The rethrow is correct (FK violations / disk full / schema drift should crash
loudly), but the verbose ERROR logging pretended the error was recoverable.
Reduced to one INFO line + rethrow.
7.2: ResponseProcessor's memorySessionId guard was throwing if the SDK hadn't
included session_id on the first user-yield, terminal-failing the entire batch.
Now warns and re-pends in-flight messages via sessionManager.markMessageFailed
(which wakes the iterator — Phase 1.2). The next iteration tries again with
memorySessionId hopefully captured.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(sync): mirror builds to installed-version cache for hot reload
When package.json bumps past Claude Code's installed pin, sync-marketplace
wrote new code to cache/<buildVersion>/ but the worker loaded from
cache/<installedVersion>/, so worker:restart reloaded the same old code.
Replace the exit-on-mismatch preflight with a mirror step: when versions
differ, also rsync plugin/ into cache/<installedVersion>/ so worker:restart
hot-reloads new code without a Claude Code session restart. The
build-version cache still gets written for the eventual
`claude plugin update`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: delete dead barrel files and orphan utilities
- src/sdk/index.ts (re-exports parser+prompts; nothing imported the barrel)
- src/services/Context.ts (re-exports ./context/index.js; no importers)
- src/services/integrations/index.ts (no importers)
- src/services/worker/Search.ts (3-line barrel of ./search/index.js)
- src/services/infrastructure/index.ts: drop CleanupV12_4_3 re-export
- src/utils/error-messages.ts (getWorkerRestartInstructions never imported)
- src/types/transcript.ts (170 LoC of types, zero importers)
- src/npx-cli/_preview.ts (banner dev preview, no script wires it)
Build + tests still pass; observations still flowing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(parser): drop unused detectLanguage
Only the user-grammar-aware variant detectLanguageWithUserGrammars()
is actually called.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(types): drop unused SdkSessionRecord + ObservationWithContext
Both interfaces in src/types/database.ts had zero importers anywhere
in src or tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(npx-cli): drop unused getDetectedIDEs + claudeMemDataDirectory
getDetectedIDEs has no callers — install.ts uses detectInstalledIDEs
directly. claudeMemDataDirectory has no callers either.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(ProcessManager): drop dead orphan-reaper + signal-handler helpers
Each had zero callers in src/ or tests/:
- cleanupOrphanedProcesses + enumerateOrphanedProcesses
- ORPHAN_PROCESS_PATTERNS + ORPHAN_MAX_AGE_MINUTES
- forceKillProcess
- waitForProcessesExit
- createSignalHandler
- resetWorkerRuntimePathCache
The orphan reaper was retired in PATHFINDER Plan 02 ("OS process groups
replace hand-rolled reapers", commit 94d592f2) — these were the leftover
pieces. shutdown.ts uses the supervisor's own kill-pgid path instead.
parseElapsedTime kept (covered by tests/infrastructure/process-manager.test.ts).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(scripts): delete 11 unreferenced DX/forensic scripts
None of these are referenced by package.json npm scripts or docs/.
All last touched on Apr 29 only as part of the comment-stripping
pass — the feature code itself is older and orphaned:
analyze-transformations-smart.js
debug-transcript-structure.ts
dump-transcript-readable.ts
endless-mode-token-calculator.js
extract-prompts-to-yaml.cjs
extract-rich-context-examples.ts
find-silent-failures.sh
fix-all-timestamps.ts
format-transcript-context.ts
test-transcript-parser.ts
transcript-to-markdown.ts
These are standalone tools — runtime behavior unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(scripts): delete unused extraction/ and types/ subdirs
- scripts/extraction/{extract-all-xml.py, filter-actual-xml.py, README.md}
point at ~/Scripts/claude-mem/ — the user's pre-relocation path that no
longer exists. Zero references in package.json, src/, or tests/.
- scripts/types/export.ts duplicates ObservationRecord etc. and has no
importers (CodexCliInstaller imports transcripts/types, not this).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(BranchManager): drop dead getInstalledPluginPath
OpenCodeInstaller has its own (used) getInstalledPluginPath; the
BranchManager copy never had any external callers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(ChromaSyncState): unexport DocKind (used internally only)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(gemini): drop stale earliestPendingTimestamp / processingMessageIds
Both fields were removed from ActiveSession in earlier queue-engine
cleanup. Tests had been silently keeping them because the mock sessions
use 'as any' to bypass strict typing, so the dead fields rode along
without complaint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: drop 3 unused module-level constants
- src/npx-cli/banner.ts: CURSOR_HOME, CLEAR_DOWN (banner uses
CLEAR_SCREEN which combines clear-down + cursor-home into a single
CSI sequence; the standalone constants were leftovers).
- src/services/worker/BranchManager.ts: DEFAULT_SHELL_TIMEOUT_MS
(BranchManager only uses GIT_COMMAND_TIMEOUT_MS / NPM_INSTALL_TIMEOUT_MS).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(opencode-plugin): drop dead workerPost helper
Only the fire-and-forget variant (workerPostFireAndForget) is actually
called. workerPost was the await-result version with no remaining caller.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: drop 8 truly-unused interface fields
Verified each by grepping for `.field`, `"field"`, `'field'`, and
`field:` patterns across src/ + tests/ + plugin/scripts. Where the
only remaining usage was the assignment site, removed the assignments too.
- GitHubStarsData: watchers_count, forks_count (only stargazers_count read)
- TableColumnInfo: dflt_value (PRAGMA returns it but no caller reads it)
- IndexInfo: seq (PRAGMA returns it but no caller reads it)
- ObservationRecord: source_files (legacy field, no readers)
- HookResult.hookSpecificOutput: permissionDecisionReason
- WatchTarget: rescanIntervalMs (set in config, never read)
- ShutdownResult: confirmedStopped (write-only — assigned but no
reader; updated all 3 return sites to drop it)
- ModePrompts: language_instruction (multilingual support never wired)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(npx-cli): reuse InstallOptions type instead of inline duplicate
parseInstallOptions had its return type written out inline as an
anonymous duplicate of InstallOptions. Use the canonical type
(import type — zero bundle cost).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(integrations): drop unused Platform type alias
The detectPlatform() function that returned this type was deleted earlier
in the branch (along with getScriptExtension that consumed it). The type
itself outlived its consumer; only string literals "Platform:" survive in
console.log diagnostics, which don't reference the alias.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(worker): broadcast processing_status when summarize is queued
broadcastSummarizeQueued was an empty no-op even though
handleSummarizeByClaudeId calls it after enqueueing. The PendingMessageStore
onMutate callback already fires broadcastProcessingStatus on enqueue, but
calling it explicitly from broadcastSummarizeQueued ensures the spinner
ticks on the moment a summary is requested even if the onMutate chain has
any timing race.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(worker): keep spinner on while summary generates
ClaudeProvider's SDK can pull multiple synthetic prompts (e.g.
observation + summarize) before producing responses. Each pull pushed
an ID to session.processingMessageIds. When the SDK's first
observation response came back, ResponseProcessor.confirmProcessed
deleted ALL pending message rows — including the still-in-flight
summary — so getTotalQueueDepth dropped to 0 and the spinner turned
off, even though the summary took another ~22s to actually generate.
Tag each in-flight message with its type ({id, type}) so the response
processor can pop only the FIFO message of the matching type
(observation vs summarize). The summary row stays in 'processing'
until its own response arrives, keeping the spinner lit through the
entire summary window.
Also updates Gemini/OpenRouter providers and GeneratorExitHandler for
the new shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(worker): clear summary from queue on any SDK response
Switch ResponseProcessor from type-aware FIFO matching to strict FIFO
popping (each SDK response → 1 in-flight message consumed). This way
the summary always clears when the SDK responds, even when the
response is unparseable or the summary doesn't actually generate
content — preventing stuck spinner / queue-depth-stuck-at-1.
Spinner behavior is preserved: messages enqueued after the summary
keep the queue depth elevated, and only when the SDK has responded
to every prompt does the queue drain to zero.
Also: when the consumed message is a 'summarize' and parsing fails,
treat it as best-effort and confirmProcessed (no retry) — summaries
that can't be parsed shouldn't keep retrying.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(viewer): redesign welcome card and remove source filters
The first-start welcome card now explains the three feed card types
(observation/summary/prompt) with color-coded badges, points users at
the gear icon for settings and the project dropdown for filtering, and
plugs /mem-search for recall — replacing the old two-line "ask:" prompts.
Source filter tabs (Claude/Codex/etc.) are removed from the header.
Filtering by AI provider was nonsense from a user POV; the project
dropdown is the only header filter now. Source tracking is also
stripped from useSSE, usePagination, App state, and CSS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(viewer): keep welcome card in feed column, swap rows for 3 squares
Two visible problems in the previous design: the card stretched
edge-to-edge while feed cards sit in a centered 650px column, and
the body was a stack of long horizontal rows that scanned line-by-line.
Both fixed: Feed now accepts a pinnedTop slot so the welcome card
renders inside the same .feed-content column as observation cards.
Body is now a 3-column grid of square feature blocks — Live feed,
Tune it, Recall it — each with a custom inline SVG illustration
(stacked cards with color-coded stripes, gear+sliders, magnifier
over cards). Old text-row sections (welcome-card-types,
welcome-card-tips, welcome-card-section, welcome-card-tip-icon)
are removed. Squares stack to one column under 600px.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(viewer): convert welcome card to glassy modal with stylized logo
Card now opens as a centered modal with a frosted/glass backdrop
(blur + saturate) so it doubles as a proper help dialog when reopened
from the header's question-mark button. Removed the observation count,
project count, and "since" date — those don't make sense for a
first-launch surface and felt out of place in a help context.
Header art swapped from the small webp logomark to the new
high-resolution sun/sunburst PNG (claude-mem-logo-stylized.png),
shipped as a checked-in asset in src/ui and plugin/ui.
Bigger throughout: 28px h2, 16px tagline, 88px illustrations,
26px feature padding, 1:1 aspect-ratio squares. Backdrop click and
Esc both close. Mobile collapses the grid to one column and drops
the aspect-ratio constraint.
Reverted the unused pinnedTop slot on Feed.tsx since the welcome
card is now a true overlay rather than an in-feed pinned card.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(viewer): make welcome modal actually glassy
Previous version had a 55%-opacity black backdrop that almost fully
blocked the underlying UI — the "glass" was just a dark plate.
Now the backdrop is fully transparent (no darkening at all), the
panel itself drops to 55% bg-card opacity with its existing
backdrop-filter blur(28px) saturate(170%), and the feature squares
drop to 35% bg-tertiary so they layer as glass-on-glass over the
already-blurred panel. The header and feed below now read clearly
through the modal's frosted blur.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(viewer): bulletproof square features via padding-bottom + clamp() fluid type
Squares were rendering taller than wide because aspect-ratio is treated
as a minimum — content can push the box past 1:1. Switched to the
classic padding-bottom: 100% trick: percentage padding resolves against
the parent's width, so the box is ALWAYS W × W regardless of content.
Inner content sits in an absolutely-positioned flex column that can't
push the shell taller.
Whole modal is now desktop-first and fluid via clamp() — no media-query
stair-steps for type, padding, gaps, border-radius, illustration size,
or modal width. Single mobile breakpoint at <600px collapses the grid
to one column and reverts the padding-bottom trick so each feature can
grow to natural content height.
Tightened the three feature descriptions so they fit comfortably inside
the square at the desktop size.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style(viewer): 15% black overlay + heavier modal shadow for elevation
Backdrop goes from transparent to rgba(0,0,0,0.15) — just enough
darkening to push the modal visually forward without burying the
underlying UI. Modal shadow stacked: 40px/120px ambient + 16px/48px
contact, both deeper, plus the existing inset 1px highlight.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(build): clear pending_messages queue on build-and-sync
Rewrites scripts/clear-failed-queue.ts to talk directly to SQLite via
bun:sqlite — the previous HTTP endpoints (/api/pending-queue/*) were
removed during the queue engine rewrite, so the script was orphaned.
Wires `npm run queue:clear` into `build-and-sync` so each rebuild
starts with a clean queue.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(worker): collapse parser to binary valid/invalid + clearPendingForSession model
- Parser: { valid: true, observations, summary } | { valid: false } — drops kind/skipped enum dispatch
- ResponseProcessor: two branches only (parseable → store + clearPendingForSession; else → no-op)
- Drop processingMessageIds + per-message claim/confirm/markFailed lifecycle across 3 providers
- PendingMessageStore: 226 → 140 lines; remove markFailed/transitionMessagesTo/confirmProcessed/clearFailedOlderThan/getAllPending/peekPendingTypes... wait keep peekPendingTypes
- Schema migration v31+v32: drop retry_count, failed_at_epoch, completed_at_epoch, worker_pid columns
- SessionQueueProcessor: delete two 1s recovery sleeps (let iterator end on error)
- Server.ts/SettingsRoutes.ts: replace four magic-number setTimeout exit-flush patterns with flushResponseThen helper
- GeneratorExitHandler: 183 → 117 lines (drain in-flight loop gone)
Net: -181 lines. No more silent data loss via maxRetries=3.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pr-2255): address review comments batch 1
- install.ts: needsMarketplace true when claude-code selected (P1, was no-op)
- install.ts: throw on invalid --model so CLI exits non-zero
- install.ts: skip worker health checks + adapt next-step copy when --no-auto-start
- install.ts: repair regenerates plugin cache when missing
- index.ts: readFlag rejects missing/flag-shaped values
- index.ts: route flag-first invocations (e.g. `--provider claude`) to install
- banner.ts: fail-open if frame payload decode throws
- SearchRoutes.ts: 5s TTL cache for settings reads on hot hook path (P2)
- detect-error-handling-antipatterns.ts: trailing-brace strip whitespace-tolerant
- investigate-timestamps.ts: compute Dec 2025 epochs at runtime (was Dec 2024)
- regenerate-claude-md.ts: include workingDir in fallback walker so root is covered
- sync-marketplace.cjs: parseWorkerPort validates 1..65535 before http.request
- sync-to-marketplace.sh: resolve SOURCE_DIR from script location, not cwd
- Dockerfile.test-installer: bash --login sources .bashrc via .bash_profile
- docs/configuration.mdx: drop nonexistent .worker.port file refs, use settings.json
- docs/architecture-overview.md: dynamic port + queue model after parser collapse
- docs/architecture/worker-service.mdx: dynamic port example + drop port-file claim
- docs/platform-integration.mdx: WORKER_BASE_URL pattern, drop hardcoded 37777
- install/public/install.sh: Node 20 floor (was 18) to match docs
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pr-2255): reset claimed messages to pending on early-return paths
ResponseProcessor returns early in two cases:
- parser invalid (unparseable response)
- memorySessionId not yet captured
Both paths previously left the just-claimed message in `status='processing'`,
which counts toward `getPendingCount`. The generator-exit handler then sees
`pendingCount > 0` and respawns the generator, looping until the restart
guard trips and `clearPendingForSession` deletes the message — silent data
loss.
Calling `resetProcessingToPending` on these paths lets the next generator
pass re-claim the message and try again, instead of burning the restart
budget on no-op respawns.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pr-2255): swebench fallback row + troubleshooting port path
- evals/swebench/run-batch.py: append fallback prediction row when
orchestrator future raises, preserving "never drop an instance" guarantee
- docs/troubleshooting.mdx: drop nonexistent .worker.port / worker.port file
references; use settings.json + /api/health for port discovery
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pr-2255): memoize per-project observation count for welcome-hint hot path
handleContextInject runs on every PostToolUse hook (after every Read/Edit).
The welcome-hint block ran a COUNT(*) on observations for every call once
CLAUDE_MEM_WELCOME_HINT_ENABLED was true. Observation counts are
monotonically increasing — once a project has any observations it always
will — so cache the positive result in a Set and skip the COUNT(*) on
subsequent requests.
Combined with the 5s settings TTL added earlier, the steady-state cost on
the hook hot path drops to a Set lookup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pr-2255): use clearProcessingForSession on AI-success path
clearPendingForSession deletes ALL rows for the session. On the success
path of processAgentResponse, that's wrong: messages that arrived as
'pending' during the (1-5s) AI response latency get deleted along with
the 'processing' row we just consumed. In a hook burst (three quick
PostToolUse hooks), B and C land while A is in flight; A's success then
nukes B and C — silent data loss.
Add a status-scoped clearProcessingForSession to PendingMessageStore +
SessionManager, and use it in ResponseProcessor's success path. The
unconditional clearPendingForSession remains correct in
GeneratorExitHandler for hard-stop / restart-guard-trip paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Revert "fix(pr-2255): use clearProcessingForSession on AI-success path"
This reverts commit a08995299c30cbad36bddc3e5bddda7af8604b35.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alex Newman
@@ -1,410 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import fs from 'fs';
|
||||
import { Database } from 'bun:sqlite';
|
||||
import readline from 'readline';
|
||||
import path from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { globSync } from 'glob';
|
||||
|
||||
// =============================================================================
|
||||
// TOOL REPLACEMENT DECISION TABLE
|
||||
// =============================================================================
|
||||
//
|
||||
// KEY INSIGHT: Observations are the SEMANTIC SYNTHESIS of tool results.
|
||||
// They contain what Claude LEARNED, which is what future Claude needs.
|
||||
//
|
||||
// Tool | Replace OUTPUT? | Reason
|
||||
// ------------------|-----------------|----------------------------------------
|
||||
// Read | ✅ YES | Observation = what was learned from file
|
||||
// Bash | ✅ YES | Observation = what command revealed
|
||||
// Grep | ✅ YES | Observation = what search found
|
||||
// Task | ✅ YES | Observation = what agent discovered
|
||||
// WebFetch | ✅ YES | Observation = what page contained
|
||||
// Glob | ⚠️ MAYBE | File lists are often small already
|
||||
// WebSearch | ⚠️ MAYBE | Results are moderate size
|
||||
// Edit | ❌ NO | OUTPUT is tiny ("success"), INPUT is ground truth
|
||||
// Write | ❌ NO | OUTPUT is tiny, INPUT is the file content
|
||||
// NotebookEdit | ❌ NO | OUTPUT is tiny, INPUT is the code
|
||||
// TodoWrite | ❌ NO | Both tiny
|
||||
// AskUserQuestion | ❌ NO | Both small, user input matters
|
||||
// mcp__* | ⚠️ MAYBE | Varies by tool
|
||||
//
|
||||
// NEVER REPLACE INPUT - it contains the action (diff, command, query, path)
|
||||
// ONLY REPLACE OUTPUT - swap raw results for semantic synthesis (observation)
|
||||
//
|
||||
// REPLACEMENT FORMAT:
|
||||
// Original output gets replaced with:
|
||||
// "[Strategically Omitted by Claude-Mem to save tokens]
|
||||
//
|
||||
// [Observation: Title here]
|
||||
// Facts: ...
|
||||
// Concepts: ..."
|
||||
// =============================================================================
|
||||
|
||||
// Configuration
|
||||
const DB_PATH = path.join(homedir(), '.claude-mem', 'claude-mem.db');
|
||||
const MAX_TRANSCRIPTS = parseInt(process.env.MAX_TRANSCRIPTS || '500', 10);
|
||||
|
||||
// Find transcript files (most recent first)
|
||||
const TRANSCRIPT_DIR = path.join(homedir(), '.claude/projects/-Users-alexnewman-Scripts-claude-mem');
|
||||
const allTranscriptFiles = globSync(path.join(TRANSCRIPT_DIR, '*.jsonl'));
|
||||
|
||||
// Sort by modification time (most recent first), take MAX_TRANSCRIPTS
|
||||
const transcriptFiles = allTranscriptFiles
|
||||
.map(f => ({ path: f, mtime: fs.statSync(f).mtime }))
|
||||
.sort((a, b) => b.mtime - a.mtime)
|
||||
.slice(0, MAX_TRANSCRIPTS)
|
||||
.map(f => f.path);
|
||||
|
||||
console.log(`Config: MAX_TRANSCRIPTS=${MAX_TRANSCRIPTS}`);
|
||||
console.log(`Using ${transcriptFiles.length} most recent transcript files (of ${allTranscriptFiles.length} total)\n`);
|
||||
|
||||
// Map to store original content from transcript (both inputs and outputs)
|
||||
const originalContent = new Map();
|
||||
|
||||
// Track contaminated (already transformed) transcripts
|
||||
let skippedTranscripts = 0;
|
||||
|
||||
// Marker for already-transformed content (endless mode replacement format)
|
||||
const TRANSFORMATION_MARKER = '**Key Facts:**';
|
||||
|
||||
// Auto-discover agent transcripts linked to main session
|
||||
async function discoverAgentFiles(mainTranscriptPath) {
|
||||
console.log('Discovering linked agent transcripts...');
|
||||
|
||||
const agentIds = new Set();
|
||||
const fileStream = fs.createReadStream(mainTranscriptPath);
|
||||
const rl = readline.createInterface({
|
||||
input: fileStream,
|
||||
crlfDelay: Infinity
|
||||
});
|
||||
|
||||
for await (const line of rl) {
|
||||
if (!line.includes('agentId')) continue;
|
||||
|
||||
try {
|
||||
const obj = JSON.parse(line);
|
||||
|
||||
// Check for agentId in toolUseResult
|
||||
if (obj.toolUseResult?.agentId) {
|
||||
agentIds.add(obj.toolUseResult.agentId);
|
||||
}
|
||||
} catch (e) {
|
||||
// Skip malformed lines
|
||||
}
|
||||
}
|
||||
|
||||
// Build agent file paths
|
||||
const directory = path.dirname(mainTranscriptPath);
|
||||
const agentFiles = Array.from(agentIds).map(id =>
|
||||
path.join(directory, `agent-${id}.jsonl`)
|
||||
).filter(filePath => fs.existsSync(filePath));
|
||||
|
||||
console.log(` → Found ${agentIds.size} agent IDs`);
|
||||
console.log(` → ${agentFiles.length} agent files exist on disk\n`);
|
||||
|
||||
return agentFiles;
|
||||
}
|
||||
|
||||
// Parse transcript to get BOTH tool_use (inputs) and tool_result (outputs) content
|
||||
// Returns true if transcript is clean, false if contaminated (already transformed)
|
||||
async function loadOriginalContentFromFile(filePath, fileLabel) {
|
||||
const fileStream = fs.createReadStream(filePath);
|
||||
const rl = readline.createInterface({
|
||||
input: fileStream,
|
||||
crlfDelay: Infinity
|
||||
});
|
||||
|
||||
let count = 0;
|
||||
let isContaminated = false;
|
||||
const toolUseIdsFromThisFile = new Set();
|
||||
|
||||
for await (const line of rl) {
|
||||
if (!line.includes('toolu_')) continue;
|
||||
|
||||
try {
|
||||
const obj = JSON.parse(line);
|
||||
|
||||
if (obj.message?.content) {
|
||||
for (const item of obj.message.content) {
|
||||
// Capture tool_use (inputs)
|
||||
if (item.type === 'tool_use' && item.id) {
|
||||
const existing = originalContent.get(item.id) || { input: '', output: '', name: '' };
|
||||
existing.input = JSON.stringify(item.input || {});
|
||||
existing.name = item.name;
|
||||
originalContent.set(item.id, existing);
|
||||
toolUseIdsFromThisFile.add(item.id);
|
||||
count++;
|
||||
}
|
||||
|
||||
// Capture tool_result (outputs)
|
||||
if (item.type === 'tool_result' && item.tool_use_id) {
|
||||
const content = typeof item.content === 'string' ? item.content : JSON.stringify(item.content);
|
||||
|
||||
// Check for transformation marker - if found, transcript is contaminated
|
||||
if (content.includes(TRANSFORMATION_MARKER)) {
|
||||
isContaminated = true;
|
||||
}
|
||||
|
||||
const existing = originalContent.get(item.tool_use_id) || { input: '', output: '', name: '' };
|
||||
existing.output = content;
|
||||
originalContent.set(item.tool_use_id, existing);
|
||||
toolUseIdsFromThisFile.add(item.tool_use_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
// Skip malformed lines
|
||||
}
|
||||
}
|
||||
|
||||
// If contaminated, remove all data from this file and report
|
||||
if (isContaminated) {
|
||||
for (const id of toolUseIdsFromThisFile) {
|
||||
originalContent.delete(id);
|
||||
}
|
||||
console.log(` ⚠️ Skipped ${fileLabel} (already transformed)`);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (count > 0) {
|
||||
console.log(` → Found ${count} tool uses in ${fileLabel}`);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
async function loadOriginalContent() {
|
||||
console.log('Loading original content from transcripts...');
|
||||
console.log(` → Scanning ${transcriptFiles.length} transcript files...\n`);
|
||||
|
||||
let cleanTranscripts = 0;
|
||||
|
||||
// Load from all transcript files
|
||||
for (const transcriptFile of transcriptFiles) {
|
||||
const filename = path.basename(transcriptFile);
|
||||
const isClean = await loadOriginalContentFromFile(transcriptFile, filename);
|
||||
if (isClean) {
|
||||
cleanTranscripts++;
|
||||
} else {
|
||||
skippedTranscripts++;
|
||||
}
|
||||
}
|
||||
|
||||
// Also check for any agent files not already included
|
||||
for (const transcriptFile of transcriptFiles) {
|
||||
if (transcriptFile.includes('agent-')) continue; // Already an agent file
|
||||
const agentFiles = await discoverAgentFiles(transcriptFile);
|
||||
for (const agentFile of agentFiles) {
|
||||
if (transcriptFiles.includes(agentFile)) continue; // Already processed
|
||||
const filename = path.basename(agentFile);
|
||||
const isClean = await loadOriginalContentFromFile(agentFile, `agent transcript (${filename})`);
|
||||
if (!isClean) {
|
||||
skippedTranscripts++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`\nTotal: Loaded original content for ${originalContent.size} tool uses (inputs + outputs)`);
|
||||
if (skippedTranscripts > 0) {
|
||||
console.log(`⚠️ Skipped ${skippedTranscripts} transcripts (already transformed with endless mode)`);
|
||||
}
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Strip __N suffix from tool_use_id to get base ID
|
||||
function getBaseToolUseId(id) {
|
||||
return id ? id.replace(/__\d+$/, '') : id;
|
||||
}
|
||||
|
||||
// Query observations from database using tool_use_ids found in transcripts
|
||||
// Handles suffixed IDs like toolu_abc__1, toolu_abc__2 matching transcript's toolu_abc
|
||||
function queryObservations() {
|
||||
// Get tool_use_ids from the loaded transcript content
|
||||
const toolUseIds = Array.from(originalContent.keys());
|
||||
|
||||
if (toolUseIds.length === 0) {
|
||||
console.log('No tool use IDs found in transcripts\n');
|
||||
return [];
|
||||
}
|
||||
|
||||
console.log(`Querying observations for ${toolUseIds.length} tool use IDs from transcripts...`);
|
||||
|
||||
const db = new Database(DB_PATH, { readonly: true });
|
||||
|
||||
// Build LIKE clauses to match both exact IDs and suffixed variants (toolu_abc, toolu_abc__1, etc)
|
||||
const likeConditions = toolUseIds.map(() => 'tool_use_id LIKE ?').join(' OR ');
|
||||
const likeParams = toolUseIds.map(id => `${id}%`);
|
||||
|
||||
const query = `
|
||||
SELECT
|
||||
id,
|
||||
tool_use_id,
|
||||
type,
|
||||
narrative,
|
||||
title,
|
||||
facts,
|
||||
concepts,
|
||||
LENGTH(COALESCE(facts,'')) as facts_len,
|
||||
LENGTH(COALESCE(title,'')) + LENGTH(COALESCE(facts,'')) as title_facts_len,
|
||||
LENGTH(COALESCE(title,'')) + LENGTH(COALESCE(facts,'')) + LENGTH(COALESCE(concepts,'')) as compact_len,
|
||||
LENGTH(COALESCE(narrative,'')) as narrative_len,
|
||||
LENGTH(COALESCE(title,'')) + LENGTH(COALESCE(narrative,'')) + LENGTH(COALESCE(facts,'')) + LENGTH(COALESCE(concepts,'')) as full_obs_len
|
||||
FROM observations
|
||||
WHERE ${likeConditions}
|
||||
ORDER BY created_at DESC
|
||||
`;
|
||||
|
||||
const observations = db.prepare(query).all(...likeParams);
|
||||
db.close();
|
||||
|
||||
console.log(`Found ${observations.length} observations matching tool use IDs (including suffixed variants)\n`);
|
||||
|
||||
return observations;
|
||||
}
|
||||
|
||||
// Tools eligible for OUTPUT replacement (observation = semantic synthesis of result)
|
||||
const REPLACEABLE_TOOLS = new Set(['Read', 'Bash', 'Grep', 'Task', 'WebFetch', 'Glob', 'WebSearch']);
|
||||
|
||||
// Analyze OUTPUT-only replacement for eligible tools
|
||||
function analyzeTransformations(observations) {
|
||||
console.log('='.repeat(110));
|
||||
console.log('OUTPUT REPLACEMENT ANALYSIS (Eligible Tools Only)');
|
||||
console.log('='.repeat(110));
|
||||
console.log();
|
||||
console.log('Eligible tools:', Array.from(REPLACEABLE_TOOLS).join(', '));
|
||||
console.log();
|
||||
|
||||
// Group observations by BASE tool_use_id (strip __N suffix)
|
||||
// This groups toolu_abc, toolu_abc__1, toolu_abc__2 together
|
||||
const obsByToolId = new Map();
|
||||
observations.forEach(obs => {
|
||||
const baseId = getBaseToolUseId(obs.tool_use_id);
|
||||
if (!obsByToolId.has(baseId)) {
|
||||
obsByToolId.set(baseId, []);
|
||||
}
|
||||
obsByToolId.get(baseId).push(obs);
|
||||
});
|
||||
|
||||
// Define strategies to test
|
||||
const strategies = [
|
||||
{ name: 'facts_only', field: 'facts_len', desc: 'Facts only (~400 chars)' },
|
||||
{ name: 'title_facts', field: 'title_facts_len', desc: 'Title + Facts (~450 chars)' },
|
||||
{ name: 'compact', field: 'compact_len', desc: 'Title + Facts + Concepts (~500 chars)' },
|
||||
{ name: 'narrative', field: 'narrative_len', desc: 'Narrative only (~700 chars)' },
|
||||
{ name: 'full', field: 'full_obs_len', desc: 'Full observation (~1200 chars)' }
|
||||
];
|
||||
|
||||
// Track results per strategy
|
||||
const results = {};
|
||||
strategies.forEach(s => {
|
||||
results[s.name] = {
|
||||
transforms: 0,
|
||||
noTransform: 0,
|
||||
saved: 0,
|
||||
totalOriginal: 0
|
||||
};
|
||||
});
|
||||
|
||||
// Track stats
|
||||
let eligible = 0;
|
||||
let ineligible = 0;
|
||||
let noTranscript = 0;
|
||||
const toolCounts = {};
|
||||
|
||||
// Analyze each tool use
|
||||
obsByToolId.forEach((obsArray, toolUseId) => {
|
||||
const original = originalContent.get(toolUseId);
|
||||
const toolName = original?.name || 'unknown';
|
||||
const outputLen = original?.output?.length || 0;
|
||||
|
||||
// Skip if no transcript data
|
||||
if (!original || outputLen === 0) {
|
||||
noTranscript++;
|
||||
return;
|
||||
}
|
||||
|
||||
// Skip if tool not eligible for replacement
|
||||
if (!REPLACEABLE_TOOLS.has(toolName)) {
|
||||
ineligible++;
|
||||
return;
|
||||
}
|
||||
|
||||
eligible++;
|
||||
toolCounts[toolName] = (toolCounts[toolName] || 0) + 1;
|
||||
|
||||
// Sum lengths across ALL observations for this tool use (handles multiple obs per tool_use_id)
|
||||
// Test each strategy - OUTPUT replacement only
|
||||
strategies.forEach(strategy => {
|
||||
const obsLen = obsArray.reduce((sum, obs) => sum + (obs[strategy.field] || 0), 0);
|
||||
const r = results[strategy.name];
|
||||
|
||||
r.totalOriginal += outputLen;
|
||||
|
||||
if (obsLen > 0 && obsLen < outputLen) {
|
||||
r.transforms++;
|
||||
r.saved += (outputLen - obsLen);
|
||||
} else {
|
||||
r.noTransform++;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// Print results
|
||||
console.log('TOOL BREAKDOWN:');
|
||||
Object.entries(toolCounts).sort((a, b) => b[1] - a[1]).forEach(([tool, count]) => {
|
||||
console.log(` ${tool}: ${count}`);
|
||||
});
|
||||
console.log();
|
||||
console.log('-'.repeat(100));
|
||||
console.log(`Eligible tool uses: ${eligible}`);
|
||||
console.log(`Ineligible (Edit/Write/etc): ${ineligible}`);
|
||||
console.log(`No transcript data: ${noTranscript}`);
|
||||
console.log('-'.repeat(100));
|
||||
console.log();
|
||||
console.log('Strategy Transforms No Transform Chars Saved Original Size Savings %');
|
||||
console.log('-'.repeat(100));
|
||||
|
||||
strategies.forEach(strategy => {
|
||||
const r = results[strategy.name];
|
||||
const pct = r.totalOriginal > 0 ? ((r.saved / r.totalOriginal) * 100).toFixed(1) : '0.0';
|
||||
console.log(
|
||||
`${strategy.desc.padEnd(35)} ${String(r.transforms).padStart(10)} ${String(r.noTransform).padStart(12)} ${String(r.saved.toLocaleString()).padStart(13)} ${String(r.totalOriginal.toLocaleString()).padStart(15)} ${pct.padStart(8)}%`
|
||||
);
|
||||
});
|
||||
|
||||
console.log('-'.repeat(100));
|
||||
console.log();
|
||||
|
||||
// Find best strategy
|
||||
let bestStrategy = null;
|
||||
let bestSavings = 0;
|
||||
strategies.forEach(strategy => {
|
||||
if (results[strategy.name].saved > bestSavings) {
|
||||
bestSavings = results[strategy.name].saved;
|
||||
bestStrategy = strategy;
|
||||
}
|
||||
});
|
||||
|
||||
if (bestStrategy) {
|
||||
const r = results[bestStrategy.name];
|
||||
const pct = ((r.saved / r.totalOriginal) * 100).toFixed(1);
|
||||
console.log(`BEST STRATEGY: ${bestStrategy.desc}`);
|
||||
console.log(` - Transforms ${r.transforms} of ${eligible} eligible tool uses (${((r.transforms/eligible)*100).toFixed(1)}%)`);
|
||||
console.log(` - Saves ${r.saved.toLocaleString()} of ${r.totalOriginal.toLocaleString()} chars (${pct}% reduction)`);
|
||||
}
|
||||
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Main execution
|
||||
async function main() {
|
||||
await loadOriginalContent();
|
||||
const observations = queryObservations();
|
||||
analyzeTransformations(observations);
|
||||
}
|
||||
|
||||
main().catch(error => {
|
||||
console.error('Fatal error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,12 +1,4 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* Error Handling Anti-Pattern Detector
|
||||
*
|
||||
* Detects try-catch anti-patterns that cause silent failures and debugging nightmares.
|
||||
* Run this before committing code that touches error handling.
|
||||
*
|
||||
* Based on hard-learned lessons: defensive try-catch wastes 10+ hours of debugging time.
|
||||
*/
|
||||
|
||||
import { readFileSync, readdirSync, statSync } from 'fs';
|
||||
import { join, relative } from 'path';
|
||||
@@ -22,9 +14,9 @@ interface AntiPattern {
|
||||
}
|
||||
|
||||
const CRITICAL_PATHS = [
|
||||
'SDKAgent.ts',
|
||||
'GeminiAgent.ts',
|
||||
'OpenRouterAgent.ts',
|
||||
'ClaudeProvider.ts',
|
||||
'GeminiProvider.ts',
|
||||
'OpenRouterProvider.ts',
|
||||
'SessionStore.ts',
|
||||
'worker-service.ts'
|
||||
];
|
||||
@@ -56,19 +48,15 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
const relPath = relative(projectRoot, filePath);
|
||||
const isCriticalPath = CRITICAL_PATHS.some(cp => filePath.includes(cp));
|
||||
|
||||
// Detect error message string matching for type detection (line-by-line patterns)
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
const trimmed = line.trim();
|
||||
|
||||
// Check for [ANTI-PATTERN IGNORED] on the same or previous line
|
||||
const hasOverride = trimmed.includes('[ANTI-PATTERN IGNORED]') ||
|
||||
(i > 0 && lines[i - 1].includes('[ANTI-PATTERN IGNORED]'));
|
||||
const overrideMatch = (trimmed + (i > 0 ? lines[i - 1] : '')).match(/\[ANTI-PATTERN IGNORED\]:\s*(.+)/i);
|
||||
const overrideReason = overrideMatch?.[1]?.trim();
|
||||
|
||||
// CRITICAL: Error message string matching for type detection
|
||||
// Patterns like: errorMessage.includes('connection') or error.message.includes('timeout')
|
||||
const errorStringMatchPatterns = [
|
||||
/error(?:Message|\.message)\s*\.includes\s*\(\s*['"`](\w+)['"`]\s*\)/i,
|
||||
/(?:err|e)\.message\s*\.includes\s*\(\s*['"`](\w+)['"`]\s*\)/i,
|
||||
@@ -79,7 +67,6 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
const match = trimmed.match(pattern);
|
||||
if (match) {
|
||||
const matchedString = match[1];
|
||||
// Common generic patterns that are too broad
|
||||
const genericPatterns = ['error', 'fail', 'connection', 'timeout', 'not', 'invalid', 'unable'];
|
||||
const isGeneric = genericPatterns.some(gp => matchedString.toLowerCase().includes(gp));
|
||||
|
||||
@@ -106,8 +93,6 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
}
|
||||
}
|
||||
|
||||
// HIGH: Logging only error.message instead of the full error object
|
||||
// Patterns like: logger.error('X', 'Y', {}, error.message) or console.error(error.message)
|
||||
const partialErrorLoggingPatterns = [
|
||||
/logger\.(error|warn|info|debug|failure)\s*\([^)]*,\s*(?:error|err|e)\.message\s*\)/,
|
||||
/logger\.(error|warn|info|debug|failure)\s*\([^)]*\{\s*(?:error|err|e):\s*(?:error|err|e)\.message\s*\}/,
|
||||
@@ -140,8 +125,6 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
}
|
||||
}
|
||||
|
||||
// CRITICAL: Catch-all error type guessing based on message content
|
||||
// Pattern: if (errorMessage.includes('X') || errorMessage.includes('Y'))
|
||||
const multipleIncludes = trimmed.match(/(?:error(?:Message|\.message)|(?:err|e)\.message).*\.includes.*\|\|.*\.includes/i);
|
||||
if (multipleIncludes) {
|
||||
if (hasOverride && overrideReason) {
|
||||
@@ -167,7 +150,6 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
}
|
||||
}
|
||||
|
||||
// Track try-catch blocks
|
||||
let inTry = false;
|
||||
let tryStartLine = 0;
|
||||
let tryLines: string[] = [];
|
||||
@@ -180,7 +162,6 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
const line = lines[i];
|
||||
const trimmed = line.trim();
|
||||
|
||||
// Detect standalone promise empty catch: .catch(() => {})
|
||||
const emptyPromiseCatch = trimmed.match(/\.catch\s*\(\s*\(\s*\)\s*=>\s*\{\s*\}\s*\)/);
|
||||
if (emptyPromiseCatch) {
|
||||
antiPatterns.push({
|
||||
@@ -193,14 +174,11 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
});
|
||||
}
|
||||
|
||||
// Detect standalone promise catch without logging: .catch(err => ...)
|
||||
const promiseCatchMatch = trimmed.match(/\.catch\s*\(\s*(?:\(\s*)?(\w+)(?:\s*\))?\s*=>/);
|
||||
if (promiseCatchMatch && !emptyPromiseCatch) {
|
||||
// Look ahead up to 10 lines to see if there's logging in the handler body
|
||||
let catchBody = trimmed.substring(promiseCatchMatch.index || 0);
|
||||
let braceCount = (catchBody.match(/{/g) || []).length - (catchBody.match(/}/g) || []).length;
|
||||
|
||||
// Collect subsequent lines if the handler spans multiple lines
|
||||
let lookAhead = 0;
|
||||
while (braceCount > 0 && lookAhead < 10 && i + lookAhead + 1 < lines.length) {
|
||||
lookAhead++;
|
||||
@@ -224,8 +202,6 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
}
|
||||
}
|
||||
|
||||
// Detect try block start (only when NOT already inside a catch block —
|
||||
// nested try/catch inside a catch is just catch-block content)
|
||||
if (!inCatch && (trimmed.match(/^\s*try\s*{/) || trimmed.match(/}\s*try\s*{/))) {
|
||||
inTry = true;
|
||||
tryStartLine = i + 1;
|
||||
@@ -234,16 +210,13 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
continue;
|
||||
}
|
||||
|
||||
// Track try block content
|
||||
if (inTry && !inCatch) {
|
||||
tryLines.push(line);
|
||||
|
||||
// Count braces to find try block end
|
||||
const openBraces = (line.match(/{/g) || []).length;
|
||||
const closeBraces = (line.match(/}/g) || []).length;
|
||||
braceDepth += openBraces - closeBraces;
|
||||
|
||||
// Found catch
|
||||
if (trimmed.match(/}\s*catch\s*(\(|{)/)) {
|
||||
inCatch = true;
|
||||
catchStartLine = i + 1;
|
||||
@@ -253,7 +226,6 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
}
|
||||
}
|
||||
|
||||
// Track catch block
|
||||
if (inCatch) {
|
||||
catchLines.push(line);
|
||||
|
||||
@@ -261,9 +233,7 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
const closeBraces = (line.match(/}/g) || []).length;
|
||||
braceDepth += openBraces - closeBraces;
|
||||
|
||||
// Catch block ended
|
||||
if (braceDepth === 0) {
|
||||
// Analyze the try-catch block
|
||||
analyzeTryCatchBlock(
|
||||
filePath,
|
||||
relPath,
|
||||
@@ -275,7 +245,6 @@ function detectAntiPatterns(filePath: string, projectRoot: string): AntiPattern[
|
||||
antiPatterns
|
||||
);
|
||||
|
||||
// Reset
|
||||
inTry = false;
|
||||
inCatch = false;
|
||||
tryLines = [];
|
||||
@@ -300,14 +269,12 @@ function analyzeTryCatchBlock(
|
||||
const tryBlock = tryLines.join('\n');
|
||||
const catchBlock = catchLines.join('\n');
|
||||
|
||||
// CRITICAL: Empty catch block
|
||||
const catchContent = catchBlock
|
||||
.replace(/}\s*catch\s*\([^)]*\)\s*{/, '') // Remove catch signature
|
||||
.replace(/}\s*catch\s*{/, '') // Remove catch without param
|
||||
.replace(/}$/, '') // Remove closing brace
|
||||
.replace(/}\s*catch\s*\([^)]*\)\s*{/, '')
|
||||
.replace(/}\s*catch\s*{/, '')
|
||||
.replace(/}\s*$/, '')
|
||||
.trim();
|
||||
|
||||
// Check for comment-only catch blocks
|
||||
const nonCommentContent = catchContent
|
||||
.split('\n')
|
||||
.filter(line => {
|
||||
@@ -328,11 +295,9 @@ function analyzeTryCatchBlock(
|
||||
});
|
||||
}
|
||||
|
||||
// Check for [ANTI-PATTERN IGNORED] marker
|
||||
const overrideMatch = catchContent.match(/\/\/\s*\[ANTI-PATTERN IGNORED\]:\s*(.+)/i);
|
||||
const overrideReason = overrideMatch?.[1]?.trim();
|
||||
|
||||
// CRITICAL: No logging in catch block (unless explicitly approved)
|
||||
const hasLogging = catchContent.match(/logger\.(error|warn|debug|info|failure)/);
|
||||
const hasConsoleError = catchContent.match(/console\.(error|warn)/);
|
||||
const hasStderr = catchContent.match(/process\.stderr\.write/);
|
||||
@@ -361,7 +326,6 @@ function analyzeTryCatchBlock(
|
||||
}
|
||||
}
|
||||
|
||||
// HIGH: Large try block (>10 lines)
|
||||
const significantTryLines = tryLines.filter(line => {
|
||||
const t = line.trim();
|
||||
return t && !t.startsWith('//') && t !== '{' && t !== '}';
|
||||
@@ -378,7 +342,6 @@ function analyzeTryCatchBlock(
|
||||
});
|
||||
}
|
||||
|
||||
// HIGH: Generic catch without type checking
|
||||
const catchParam = catchBlock.match(/catch\s*\(([^)]+)\)/)?.[1]?.trim();
|
||||
const hasTypeCheck = catchContent.match(/instanceof\s+Error/) ||
|
||||
catchContent.match(/\.name\s*===/) ||
|
||||
@@ -395,7 +358,6 @@ function analyzeTryCatchBlock(
|
||||
});
|
||||
}
|
||||
|
||||
// CRITICAL on critical paths: Catch-and-continue
|
||||
if (isCriticalPath && nonCommentContent && !hasThrow) {
|
||||
const hasReturn = catchContent.match(/return/);
|
||||
const hasProcessExit = catchContent.match(/process\.exit/);
|
||||
@@ -486,7 +448,6 @@ function formatReport(antiPatterns: AntiPattern[]): string {
|
||||
return report;
|
||||
}
|
||||
|
||||
// Main execution
|
||||
const projectRoot = process.cwd();
|
||||
const srcDir = join(projectRoot, 'src');
|
||||
|
||||
@@ -505,7 +466,6 @@ for (const file of tsFiles) {
|
||||
const report = formatReport(allAntiPatterns);
|
||||
console.log(report);
|
||||
|
||||
// Exit with error code if any issues found
|
||||
const issues = allAntiPatterns.filter(a => a.severity === 'ISSUE');
|
||||
if (issues.length > 0) {
|
||||
console.error(`❌ FAILED: ${issues.length} error handling anti-patterns must be fixed.\n`);
|
||||
|
||||
@@ -112,12 +112,10 @@ async function promptMultiline(prompt: string): Promise<string> {
|
||||
|
||||
return new Promise((resolve) => {
|
||||
rl.on("line", (line) => {
|
||||
// Empty line means we're done
|
||||
if (line.trim() === "" && lines.length > 0) {
|
||||
rl.close();
|
||||
resolve(lines.join("\n"));
|
||||
} else if (line.trim() !== "") {
|
||||
// Only add non-empty lines (or preserve empty lines in the middle)
|
||||
lines.push(line);
|
||||
}
|
||||
});
|
||||
@@ -139,7 +137,6 @@ async function main() {
|
||||
console.log("🌎 Leave report in ANY language, and it will auto translate to English\n");
|
||||
console.log("🔍 Collecting system diagnostics...");
|
||||
|
||||
// Collect diagnostics
|
||||
const diagnostics = await collectDiagnostics({
|
||||
includeLogs: !args.noLogs,
|
||||
});
|
||||
@@ -154,7 +151,6 @@ async function main() {
|
||||
}
|
||||
console.log("✓ Configuration loaded\n");
|
||||
|
||||
// Show summary
|
||||
console.log("📋 System Summary:");
|
||||
console.log(` Claude-mem: v${diagnostics.versions.claudeMem}`);
|
||||
console.log(` Claude Code: ${diagnostics.versions.claudeCode}`);
|
||||
@@ -171,7 +167,6 @@ async function main() {
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Prompt for issue details
|
||||
const issueDescription = await promptMultiline(
|
||||
"Please describe the issue you're experiencing:"
|
||||
);
|
||||
@@ -203,7 +198,6 @@ async function main() {
|
||||
|
||||
console.log("\n🤖 Generating bug report with Claude...");
|
||||
|
||||
// Generate the bug report
|
||||
const result = await generateBugReport({
|
||||
issueDescription,
|
||||
expectedBehavior: expectedBehavior.trim() || undefined,
|
||||
@@ -218,7 +212,6 @@ async function main() {
|
||||
|
||||
console.log("✓ Issue formatted successfully\n");
|
||||
|
||||
// Generate output file path
|
||||
const timestamp = new Date()
|
||||
.toISOString()
|
||||
.replace(/:/g, "")
|
||||
@@ -230,15 +223,12 @@ async function main() {
|
||||
);
|
||||
const outputPath = args.output || defaultOutputPath;
|
||||
|
||||
// Save to file
|
||||
await fs.writeFile(outputPath, result.body, "utf-8");
|
||||
|
||||
// Build GitHub URL with pre-filled title and body
|
||||
const encodedTitle = encodeURIComponent(result.title);
|
||||
const encodedBody = encodeURIComponent(result.body);
|
||||
const githubUrl = `https://github.com/thedotmack/claude-mem/issues/new?title=${encodedTitle}&body=${encodedBody}`;
|
||||
|
||||
// Display the report
|
||||
console.log("─".repeat(60));
|
||||
console.log("📋 BUG REPORT GENERATED");
|
||||
console.log("─".repeat(60));
|
||||
@@ -251,7 +241,6 @@ async function main() {
|
||||
console.log("─".repeat(60));
|
||||
console.log();
|
||||
|
||||
// Open GitHub issue in browser
|
||||
console.log("🌐 Opening GitHub issue form in your browser...");
|
||||
try {
|
||||
const openCommand =
|
||||
|
||||
@@ -216,7 +216,6 @@ export async function collectDiagnostics(
|
||||
const cwd = process.cwd();
|
||||
const isDevMode = cwd.includes("claude-mem") && !cwd.includes(".claude");
|
||||
|
||||
// Collect version information
|
||||
const [claudeMem, claudeCode, bun, osVersion] = await Promise.all([
|
||||
getClaudememVersion(),
|
||||
getClaudeCodeVersion(),
|
||||
@@ -244,7 +243,6 @@ export async function collectDiagnostics(
|
||||
isDevMode,
|
||||
};
|
||||
|
||||
// Check worker status
|
||||
const pidInfo = await readPidFile(dataDir);
|
||||
const workerPort = pidInfo?.port || 37777;
|
||||
|
||||
@@ -263,7 +261,6 @@ export async function collectDiagnostics(
|
||||
stats,
|
||||
};
|
||||
|
||||
// Collect logs if requested
|
||||
let workerLog: string[] = [];
|
||||
let silentLog: string[] = [];
|
||||
|
||||
@@ -283,7 +280,6 @@ export async function collectDiagnostics(
|
||||
silentLog: silentLog.map(sanitizePath),
|
||||
};
|
||||
|
||||
// Database info
|
||||
const [dbInfo, tableCounts] = await Promise.all([
|
||||
getDatabaseInfo(dataDir),
|
||||
getTableCounts(dataDir),
|
||||
@@ -295,7 +291,6 @@ export async function collectDiagnostics(
|
||||
counts: tableCounts,
|
||||
};
|
||||
|
||||
// Configuration
|
||||
const settingsInfo = await getSettings(dataDir);
|
||||
const config = {
|
||||
settingsPath: sanitizePath(path.join(dataDir, "settings.json")),
|
||||
@@ -381,7 +376,6 @@ export function formatDiagnostics(diagnostics: SystemDiagnostics): string {
|
||||
}
|
||||
output += "\n";
|
||||
|
||||
// Add logs if present
|
||||
if (diagnostics.logs.workerLog.length > 0) {
|
||||
output += "## Recent Worker Logs (Last 50 Lines)\n\n";
|
||||
output += "```\n";
|
||||
|
||||
@@ -27,14 +27,12 @@ export async function generateBugReport(
|
||||
input: BugReportInput
|
||||
): Promise<BugReportResult> {
|
||||
try {
|
||||
// Collect system diagnostics
|
||||
const diagnostics = await collectDiagnostics({
|
||||
includeLogs: input.includeLogs !== false,
|
||||
});
|
||||
|
||||
const formattedDiagnostics = formatDiagnostics(diagnostics);
|
||||
|
||||
// Build the prompt
|
||||
const prompt = buildPrompt(
|
||||
formattedDiagnostics,
|
||||
input.issueDescription,
|
||||
@@ -42,7 +40,6 @@ export async function generateBugReport(
|
||||
input.stepsToReproduce
|
||||
);
|
||||
|
||||
// Use Agent SDK to generate formatted issue
|
||||
let generatedMarkdown = "";
|
||||
let charCount = 0;
|
||||
const startTime = Date.now();
|
||||
@@ -58,11 +55,9 @@ export async function generateBugReport(
|
||||
},
|
||||
});
|
||||
|
||||
// Progress spinner frames
|
||||
const spinnerFrames = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"];
|
||||
let spinnerIdx = 0;
|
||||
|
||||
// Stream the response
|
||||
for await (const message of stream) {
|
||||
if (message.type === "stream_event") {
|
||||
const event = message.event as { type: string; delta?: { type: string; text?: string } };
|
||||
@@ -76,7 +71,6 @@ export async function generateBugReport(
|
||||
}
|
||||
}
|
||||
|
||||
// Handle full assistant messages (fallback)
|
||||
if (message.type === "assistant") {
|
||||
for (const block of message.message.content) {
|
||||
if (block.type === "text" && !generatedMarkdown) {
|
||||
@@ -86,7 +80,6 @@ export async function generateBugReport(
|
||||
}
|
||||
}
|
||||
|
||||
// Handle result
|
||||
if (message.type === "result") {
|
||||
const result = message as SDKResultMessage;
|
||||
if (result.subtype === "success" && !generatedMarkdown && result.result) {
|
||||
@@ -96,10 +89,8 @@ export async function generateBugReport(
|
||||
}
|
||||
}
|
||||
|
||||
// Clear the progress line
|
||||
process.stdout.write("\r" + " ".repeat(60) + "\r");
|
||||
|
||||
// Extract title from markdown (first heading)
|
||||
const titleMatch = generatedMarkdown.match(/^#\s+(.+)$/m);
|
||||
const title = titleMatch ? titleMatch[1] : "Bug Report";
|
||||
|
||||
@@ -109,7 +100,6 @@ export async function generateBugReport(
|
||||
success: true,
|
||||
};
|
||||
} catch (error) {
|
||||
// Fallback to template-based generation
|
||||
console.error("Agent SDK failed, using template fallback:", error);
|
||||
return generateTemplateFallback(input);
|
||||
}
|
||||
|
||||
+13
-79
@@ -1,10 +1,5 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Build script for claude-mem hooks
|
||||
* Bundles TypeScript hooks into individual standalone executables using esbuild
|
||||
*/
|
||||
|
||||
import { build } from 'esbuild';
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
@@ -27,39 +22,18 @@ const CONTEXT_GENERATOR = {
|
||||
source: 'src/services/context-generator.ts'
|
||||
};
|
||||
|
||||
/**
|
||||
* Strip hardcoded __dirname/__filename from bundled CJS output.
|
||||
*
|
||||
* When esbuild converts ESM TypeScript source to CJS format, it inlines
|
||||
* __dirname and __filename as static strings based on the SOURCE file paths
|
||||
* at build time. These `var __dirname = "/build/machine/path/..."` declarations
|
||||
* shadow the runtime's native __dirname (provided by Bun/Node's CJS module
|
||||
* wrapper), causing path resolution to fail on end-user machines.
|
||||
*
|
||||
* This post-build step removes those hardcoded assignments so the runtime
|
||||
* globals are used instead.
|
||||
*
|
||||
* See: https://github.com/thedotmack/claude-mem/issues/1410
|
||||
*/
|
||||
function stripHardcodedDirname(filePath) {
|
||||
let content = fs.readFileSync(filePath, 'utf-8');
|
||||
const before = content.length;
|
||||
|
||||
// Match both double-quoted and single-quoted string literals.
|
||||
// esbuild currently emits double quotes, but single quotes are handled
|
||||
// defensively in case future versions change quoting style.
|
||||
const str = `(?:"[^"]*"|'[^']*')`;
|
||||
|
||||
for (const id of ['__dirname', '__filename']) {
|
||||
// Remove `var <id> = "...", rest` → `var rest`
|
||||
content = content.replace(new RegExp(`\\bvar ${id}\\s*=\\s*${str},\\s*`, 'g'), 'var ');
|
||||
// Remove standalone `var <id> = "...";`
|
||||
content = content.replace(new RegExp(`\\bvar ${id}\\s*=\\s*${str};\\s*`, 'g'), '');
|
||||
// Remove `, <id> = "..."` from mid/end of var declarations
|
||||
content = content.replace(new RegExp(`,\\s*${id}\\s*=\\s*${str}`, 'g'), '');
|
||||
}
|
||||
|
||||
// Clean up dangling `var ;` left when __dirname was the sole declarator
|
||||
content = content.replace(/\bvar\s*;/g, '');
|
||||
|
||||
const removed = before - content.length;
|
||||
@@ -73,12 +47,10 @@ async function buildHooks() {
|
||||
console.log('🔨 Building claude-mem hooks and worker service...\n');
|
||||
|
||||
try {
|
||||
// Read version from package.json
|
||||
const packageJson = JSON.parse(fs.readFileSync('package.json', 'utf-8'));
|
||||
const version = packageJson.version;
|
||||
console.log(`📌 Version: ${version}`);
|
||||
|
||||
// Create output directories
|
||||
console.log('\n📦 Preparing output directories...');
|
||||
const hooksDir = 'plugin/scripts';
|
||||
const uiDir = 'plugin/ui';
|
||||
@@ -91,8 +63,6 @@ async function buildHooks() {
|
||||
}
|
||||
console.log('✓ Output directories ready');
|
||||
|
||||
// Generate plugin/package.json for cache directory dependency installation
|
||||
// Note: bun:sqlite is a Bun built-in, no external dependencies needed for SQLite
|
||||
console.log('\n📦 Generating plugin package.json...');
|
||||
const pluginPackageJson = {
|
||||
name: 'claude-mem-plugin',
|
||||
@@ -101,10 +71,6 @@ async function buildHooks() {
|
||||
description: 'Runtime dependencies for claude-mem bundled hooks',
|
||||
type: 'module',
|
||||
dependencies: {
|
||||
// Externalized from mcp-server.cjs to avoid Zod version conflicts when
|
||||
// OpenCode's Bun bundler assembles hook scripts (#2113). MCP SDK
|
||||
// transitively imports Zod; loading it via node_modules at runtime
|
||||
// ensures OpenCode controls the version.
|
||||
'zod': '^4.3.6',
|
||||
'tree-sitter-cli': '^0.26.5',
|
||||
'tree-sitter-c': '^0.24.1',
|
||||
@@ -132,11 +98,6 @@ async function buildHooks() {
|
||||
'@derekstride/tree-sitter-sql': '^0.3.11',
|
||||
'@tree-sitter-grammars/tree-sitter-markdown': '^0.3.2',
|
||||
},
|
||||
// The grammar packages above declare three different majors of `tree-sitter`
|
||||
// as peer deps (^0.21, ^0.22, ^0.25). Bun and pnpm are lenient enough to
|
||||
// pick one and move on, but plain `npm install --production` aborts with
|
||||
// ERESOLVE. Pinning a single version via `overrides` lets npm resolve a
|
||||
// working tree without `--legacy-peer-deps`. Closes #2147.
|
||||
overrides: {
|
||||
'tree-sitter': '^0.25.0'
|
||||
},
|
||||
@@ -148,7 +109,6 @@ async function buildHooks() {
|
||||
fs.writeFileSync('plugin/package.json', JSON.stringify(pluginPackageJson, null, 2) + '\n');
|
||||
console.log('✓ plugin/package.json generated');
|
||||
|
||||
// Build React viewer
|
||||
console.log('\n📋 Building React viewer...');
|
||||
const { spawn } = await import('child_process');
|
||||
const viewerBuild = spawn('node', ['scripts/build-viewer.js'], { stdio: 'inherit' });
|
||||
@@ -162,7 +122,6 @@ async function buildHooks() {
|
||||
});
|
||||
});
|
||||
|
||||
// Build worker service
|
||||
console.log(`\n🔧 Building worker service...`);
|
||||
await build({
|
||||
entryPoints: [WORKER_SERVICE.source],
|
||||
@@ -175,10 +134,8 @@ async function buildHooks() {
|
||||
logLevel: 'error', // Suppress warnings (import.meta warning is benign)
|
||||
external: [
|
||||
'bun:sqlite',
|
||||
// Optional chromadb embedding providers
|
||||
'cohere-ai',
|
||||
'ollama',
|
||||
// Default embedding function with native binaries
|
||||
'@chroma-core/default-embed',
|
||||
'onnxruntime-node'
|
||||
],
|
||||
@@ -194,15 +151,12 @@ async function buildHooks() {
|
||||
}
|
||||
});
|
||||
|
||||
// Fix hardcoded __dirname/__filename in bundled output (#1410)
|
||||
stripHardcodedDirname(`${hooksDir}/${WORKER_SERVICE.name}.cjs`);
|
||||
|
||||
// Make worker service executable
|
||||
fs.chmodSync(`${hooksDir}/${WORKER_SERVICE.name}.cjs`, 0o755);
|
||||
const workerStats = fs.statSync(`${hooksDir}/${WORKER_SERVICE.name}.cjs`);
|
||||
console.log(`✓ worker-service built (${(workerStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build MCP server
|
||||
console.log(`\n🔧 Building MCP server...`);
|
||||
await build({
|
||||
entryPoints: [MCP_SERVER.source],
|
||||
@@ -215,10 +169,6 @@ async function buildHooks() {
|
||||
logLevel: 'error',
|
||||
external: [
|
||||
'bun:sqlite',
|
||||
// Externalize Zod to avoid version conflicts when OpenCode's Bun bundler
|
||||
// assembles hook scripts (see #2113). The MCP server transitively imports
|
||||
// Zod via @modelcontextprotocol/sdk; bundling it caused two Zod versions
|
||||
// to coexist at runtime and the v4 ↔ v3 _zod.def access crashed.
|
||||
'zod',
|
||||
'tree-sitter-cli',
|
||||
'tree-sitter-javascript',
|
||||
@@ -254,24 +204,12 @@ async function buildHooks() {
|
||||
}
|
||||
});
|
||||
|
||||
// Fix hardcoded __dirname/__filename in bundled output (#1410)
|
||||
stripHardcodedDirname(`${hooksDir}/${MCP_SERVER.name}.cjs`);
|
||||
|
||||
// Make MCP server executable
|
||||
fs.chmodSync(`${hooksDir}/${MCP_SERVER.name}.cjs`, 0o755);
|
||||
const mcpServerStats = fs.statSync(`${hooksDir}/${MCP_SERVER.name}.cjs`);
|
||||
console.log(`✓ mcp-server built (${(mcpServerStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// GUARDRAIL (#1645): The MCP server runs under Node, but the entire `bun:`
|
||||
// module namespace (bun:sqlite, bun:ffi, bun:test, etc.) is Bun-only. If
|
||||
// any transitive import in mcp-server.ts ever pulls one in, the bundle
|
||||
// will crash on first require under Node — which is exactly the regression
|
||||
// PR #1645 fixed for `bun:sqlite`. Fail the build instead of shipping a
|
||||
// broken bundle so future contributors get an immediate signal.
|
||||
//
|
||||
// Only flag actual `require("bun:...")` / `require('bun:...')` calls, not
|
||||
// the bare string — error messages and inline comments may legitimately
|
||||
// mention `bun:sqlite` by name without re-introducing the import.
|
||||
const mcpBundleContent = fs.readFileSync(`${hooksDir}/${MCP_SERVER.name}.cjs`, 'utf-8');
|
||||
const bunRequireRegex = /require\(\s*["']bun:[a-z][a-z0-9_-]*["']\s*\)/;
|
||||
const bunRequireMatch = mcpBundleContent.match(bunRequireRegex);
|
||||
@@ -281,16 +219,6 @@ async function buildHooks() {
|
||||
);
|
||||
}
|
||||
|
||||
// SECONDARY GUARDRAIL (#1645 round 11): bundle size budget. The bun:sqlite
|
||||
// regex above catches the specific regression class we already know about,
|
||||
// but esbuild could in theory change how it emits external module specifiers
|
||||
// and silently slip past the regex. A bundle-size budget catches the
|
||||
// structural symptom (worker-service.ts dragged into the bundle blew the
|
||||
// size from ~358KB to ~1.96MB) regardless of how the imports look.
|
||||
//
|
||||
// 600KB is a generous ceiling — current size is ~384KB, the broken v12.0.0
|
||||
// bundle was ~1920KB, and there's plenty of headroom for legitimate growth
|
||||
// before we'd want to revisit this number.
|
||||
const MCP_SERVER_MAX_BYTES = 600 * 1024;
|
||||
if (mcpServerStats.size > MCP_SERVER_MAX_BYTES) {
|
||||
throw new Error(
|
||||
@@ -298,7 +226,6 @@ async function buildHooks() {
|
||||
);
|
||||
}
|
||||
|
||||
// Build context generator
|
||||
console.log(`\n🔧 Building context generator...`);
|
||||
await build({
|
||||
entryPoints: [CONTEXT_GENERATOR.source],
|
||||
@@ -316,13 +243,11 @@ async function buildHooks() {
|
||||
// No banner needed: CJS files under Node.js have __dirname/__filename natively
|
||||
});
|
||||
|
||||
// Fix hardcoded __dirname/__filename in bundled output (#1410)
|
||||
stripHardcodedDirname(`${hooksDir}/${CONTEXT_GENERATOR.name}.cjs`);
|
||||
|
||||
const contextGenStats = fs.statSync(`${hooksDir}/${CONTEXT_GENERATOR.name}.cjs`);
|
||||
console.log(`✓ context-generator built (${(contextGenStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build NPX CLI (pure Node.js — no Bun dependency)
|
||||
console.log(`\n🔧 Building NPX CLI...`);
|
||||
const npxCliOutDir = 'dist/npx-cli';
|
||||
if (!fs.existsSync(npxCliOutDir)) {
|
||||
@@ -342,18 +267,17 @@ async function buildHooks() {
|
||||
'fs', 'fs/promises', 'path', 'os', 'child_process', 'url',
|
||||
'crypto', 'http', 'https', 'net', 'stream', 'util', 'events',
|
||||
'buffer', 'querystring', 'readline', 'tty', 'assert',
|
||||
'bun:sqlite',
|
||||
],
|
||||
define: {
|
||||
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
|
||||
},
|
||||
});
|
||||
|
||||
// Make NPX CLI executable
|
||||
fs.chmodSync(`${npxCliOutDir}/index.js`, 0o755);
|
||||
const npxCliStats = fs.statSync(`${npxCliOutDir}/index.js`);
|
||||
console.log(`✓ npx-cli built (${(npxCliStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build OpenClaw plugin (self-contained, only Node builtins external)
|
||||
if (fs.existsSync('openclaw/src/index.ts')) {
|
||||
console.log(`\n🔧 Building OpenClaw plugin...`);
|
||||
const openclawOutDir = 'openclaw/dist';
|
||||
@@ -379,7 +303,6 @@ async function buildHooks() {
|
||||
console.log(`✓ openclaw plugin built (${(openclawStats.size / 1024).toFixed(2)} KB)`);
|
||||
}
|
||||
|
||||
// Build OpenCode plugin (self-contained, Node.js ESM — Bun-compatible)
|
||||
if (fs.existsSync('src/integrations/opencode-plugin/index.ts')) {
|
||||
console.log(`\n🔧 Building OpenCode plugin...`);
|
||||
const opencodeOutDir = 'dist/opencode-plugin';
|
||||
@@ -405,11 +328,22 @@ async function buildHooks() {
|
||||
console.log(`✓ opencode plugin built (${(opencodeStats.size / 1024).toFixed(2)} KB)`);
|
||||
}
|
||||
|
||||
// Verify critical distribution files exist (skills are source files, not build outputs)
|
||||
console.log('\n📋 Copying onboarding explainer to plugin tree...');
|
||||
const onboardingExplainerSrc = 'src/services/worker/onboarding-explainer.md';
|
||||
const onboardingExplainerDst = 'plugin/skills/how-it-works/onboarding-explainer.md';
|
||||
if (!fs.existsSync(onboardingExplainerSrc)) {
|
||||
throw new Error(`Missing onboarding explainer source: ${onboardingExplainerSrc}`);
|
||||
}
|
||||
fs.mkdirSync(path.dirname(onboardingExplainerDst), { recursive: true });
|
||||
fs.copyFileSync(onboardingExplainerSrc, onboardingExplainerDst);
|
||||
console.log(`✓ Copied ${onboardingExplainerSrc} → ${onboardingExplainerDst}`);
|
||||
|
||||
console.log('\n📋 Verifying distribution files...');
|
||||
const requiredDistributionFiles = [
|
||||
'plugin/skills/mem-search/SKILL.md',
|
||||
'plugin/skills/smart-explore/SKILL.md',
|
||||
'plugin/skills/how-it-works/SKILL.md',
|
||||
'plugin/skills/how-it-works/onboarding-explainer.md',
|
||||
'plugin/hooks/hooks.json',
|
||||
'plugin/.claude-plugin/plugin.json',
|
||||
];
|
||||
|
||||
@@ -13,7 +13,6 @@ async function buildViewer() {
|
||||
console.log('Building React viewer...');
|
||||
|
||||
try {
|
||||
// Build React app
|
||||
await esbuild.build({
|
||||
entryPoints: [path.join(rootDir, 'src/ui/viewer/index.tsx')],
|
||||
bundle: true,
|
||||
@@ -32,7 +31,6 @@ async function buildViewer() {
|
||||
}
|
||||
});
|
||||
|
||||
// Copy HTML template to build output
|
||||
const htmlTemplate = fs.readFileSync(
|
||||
path.join(rootDir, 'src/ui/viewer-template.html'),
|
||||
'utf-8'
|
||||
@@ -42,7 +40,6 @@ async function buildViewer() {
|
||||
htmlTemplate
|
||||
);
|
||||
|
||||
// Copy font assets
|
||||
const fontsDir = path.join(rootDir, 'src/ui/viewer/assets/fonts');
|
||||
const outputFontsDir = path.join(rootDir, 'plugin/ui/assets/fonts');
|
||||
|
||||
@@ -57,7 +54,6 @@ async function buildViewer() {
|
||||
}
|
||||
}
|
||||
|
||||
// Copy icon SVG files
|
||||
const srcUiDir = path.join(rootDir, 'src/ui');
|
||||
const outputUiDir = path.join(rootDir, 'plugin/ui');
|
||||
const iconFiles = fs.readdirSync(srcUiDir).filter(file => file.startsWith('icon-thick-') && file.endsWith('.svg'));
|
||||
|
||||
@@ -1,8 +1,4 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Build Windows executable for claude-mem worker service
|
||||
* Uses Bun's compile feature to create a standalone exe
|
||||
*/
|
||||
|
||||
import { execSync } from 'child_process';
|
||||
import fs from 'fs';
|
||||
|
||||
@@ -1,12 +1,4 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* Check and process pending observation queue
|
||||
*
|
||||
* Usage:
|
||||
* bun scripts/check-pending-queue.ts # Check status and prompt to process
|
||||
* bun scripts/check-pending-queue.ts --process # Auto-process without prompting
|
||||
* bun scripts/check-pending-queue.ts --limit 5 # Process up to 5 sessions
|
||||
*/
|
||||
|
||||
const WORKER_URL = 'http://localhost:37777';
|
||||
|
||||
@@ -82,7 +74,6 @@ function formatAge(epochMs: number): string {
|
||||
}
|
||||
|
||||
async function prompt(question: string): Promise<string> {
|
||||
// Check if we have a TTY for interactive input
|
||||
if (!process.stdin.isTTY) {
|
||||
console.log(question + '(no TTY, use --process flag for non-interactive mode)');
|
||||
return 'n';
|
||||
@@ -102,7 +93,6 @@ async function prompt(question: string): Promise<string> {
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
|
||||
// Help flag
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
console.log(`
|
||||
Claude-Mem Pending Queue Manager
|
||||
@@ -142,7 +132,6 @@ What is this for?
|
||||
|
||||
console.log('\n=== Claude-Mem Pending Queue Status ===\n');
|
||||
|
||||
// Check worker health
|
||||
const healthy = await checkWorkerHealth();
|
||||
if (!healthy) {
|
||||
console.log('Worker is not running. Start it with:');
|
||||
@@ -151,11 +140,9 @@ What is this for?
|
||||
}
|
||||
console.log('Worker status: Running\n');
|
||||
|
||||
// Get queue status
|
||||
const status = await getQueueStatus();
|
||||
const { queue, sessionsWithPendingWork } = status;
|
||||
|
||||
// Display summary
|
||||
console.log('Queue Summary:');
|
||||
console.log(` Pending: ${queue.totalPending}`);
|
||||
console.log(` Processing: ${queue.totalProcessing}`);
|
||||
@@ -163,26 +150,22 @@ What is this for?
|
||||
console.log(` Stuck: ${queue.stuckCount} (processing > 5 min)`);
|
||||
console.log(` Sessions: ${sessionsWithPendingWork.length} with pending work\n`);
|
||||
|
||||
// Check if there's any backlog
|
||||
const hasBacklog = queue.totalPending > 0 || queue.totalFailed > 0;
|
||||
const hasStuck = queue.stuckCount > 0;
|
||||
|
||||
if (!hasBacklog && !hasStuck) {
|
||||
console.log('No backlog detected. Queue is healthy.\n');
|
||||
|
||||
// Show recently processed if any
|
||||
if (status.recentlyProcessed.length > 0) {
|
||||
console.log(`Recently processed: ${status.recentlyProcessed.length} messages in last 30 min\n`);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Show details about pending messages
|
||||
if (queue.messages.length > 0) {
|
||||
console.log('Pending Messages:');
|
||||
console.log('─'.repeat(80));
|
||||
|
||||
// Group by session
|
||||
const bySession = new Map<number, QueueMessage[]>();
|
||||
for (const msg of queue.messages) {
|
||||
const list = bySession.get(msg.session_db_id) || [];
|
||||
@@ -208,7 +191,6 @@ What is this for?
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Offer to process
|
||||
if (autoProcess) {
|
||||
console.log(`Auto-processing up to ${limit} sessions...\n`);
|
||||
} else {
|
||||
@@ -220,7 +202,6 @@ What is this for?
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Process the queue
|
||||
const result = await processQueue(limit);
|
||||
|
||||
console.log('Processing Result:');
|
||||
|
||||
@@ -1,22 +1,4 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* Cleanup script for duplicate observations created by the batching bug.
|
||||
*
|
||||
* The bug: When multiple messages were batched together, observations were stored
|
||||
* once per message ID instead of once per observation. For example, if 4 messages
|
||||
* were batched and produced 3 observations, those 3 observations were stored
|
||||
* 12 times (4×3) instead of 3 times.
|
||||
*
|
||||
* This script identifies duplicates by matching on:
|
||||
* - memory_session_id (same session)
|
||||
* - text (same content)
|
||||
* - type (same observation type)
|
||||
* - created_at_epoch within 60 seconds (same batch window)
|
||||
*
|
||||
* Usage:
|
||||
* bun scripts/cleanup-duplicates.ts # Dry run (default)
|
||||
* bun scripts/cleanup-duplicates.ts --execute # Actually delete duplicates
|
||||
*/
|
||||
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { homedir } from 'os';
|
||||
@@ -24,7 +6,6 @@ import { join } from 'path';
|
||||
|
||||
const DB_PATH = join(homedir(), '.claude-mem', 'claude-mem.db');
|
||||
|
||||
// Time window modes for duplicate detection
|
||||
const TIME_WINDOW_MODES = {
|
||||
strict: 5, // 5 seconds - only exact duplicates from same batch
|
||||
normal: 60, // 60 seconds - duplicates within same minute
|
||||
@@ -57,7 +38,6 @@ function main() {
|
||||
const aggressive = process.argv.includes('--aggressive');
|
||||
const strict = process.argv.includes('--strict');
|
||||
|
||||
// Determine time window
|
||||
let windowMode: keyof typeof TIME_WINDOW_MODES = 'normal';
|
||||
if (aggressive) windowMode = 'aggressive';
|
||||
if (strict) windowMode = 'strict';
|
||||
@@ -80,11 +60,9 @@ function main() {
|
||||
? new Database(DB_PATH, { readonly: true })
|
||||
: new Database(DB_PATH);
|
||||
|
||||
// Get total observation count
|
||||
const totalCount = db.prepare('SELECT COUNT(*) as count FROM observations').get() as { count: number };
|
||||
console.log(`Total observations in database: ${totalCount.count}`);
|
||||
|
||||
// Find all observations and group by content fingerprint
|
||||
const observations = db.prepare(`
|
||||
SELECT
|
||||
id,
|
||||
@@ -101,23 +79,17 @@ function main() {
|
||||
console.log(`Analyzing ${observations.length} observations for duplicates...`);
|
||||
console.log('');
|
||||
|
||||
// Group observations by fingerprint (session + text + type + time bucket)
|
||||
const groups = new Map<string, ObservationRow[]>();
|
||||
|
||||
for (const obs of observations) {
|
||||
// Skip observations without title (can't dedupe without content identifier)
|
||||
if (obs.title === null) continue;
|
||||
|
||||
// Create content hash from title + subtitle + narrative
|
||||
const contentKey = `${obs.title}|${obs.subtitle || ''}|${obs.narrative || ''}`;
|
||||
|
||||
// Create fingerprint based on time window mode
|
||||
let fingerprint: string;
|
||||
if (batchWindowSeconds === 0) {
|
||||
// Aggressive mode: ignore time entirely
|
||||
fingerprint = `${obs.memory_session_id}|${obs.type}|${contentKey}`;
|
||||
} else {
|
||||
// Normal/strict mode: include time bucket
|
||||
const epochBucket = Math.floor(obs.created_at_epoch / batchWindowSeconds);
|
||||
fingerprint = `${obs.memory_session_id}|${obs.type}|${epochBucket}|${contentKey}`;
|
||||
}
|
||||
@@ -128,17 +100,14 @@ function main() {
|
||||
groups.get(fingerprint)!.push(obs);
|
||||
}
|
||||
|
||||
// Find groups with duplicates
|
||||
const duplicateGroups: DuplicateGroup[] = [];
|
||||
|
||||
for (const [fingerprint, rows] of groups) {
|
||||
if (rows.length > 1) {
|
||||
// Sort by id to keep the oldest (lowest id)
|
||||
rows.sort((a, b) => a.id - b.id);
|
||||
const keepId = rows[0].id;
|
||||
const deleteIds = rows.slice(1).map(r => r.id);
|
||||
|
||||
// SAFETY: Never delete all copies - always keep at least one
|
||||
if (deleteIds.length >= rows.length) {
|
||||
throw new Error(`SAFETY VIOLATION: Would delete all ${rows.length} copies! Aborting.`);
|
||||
}
|
||||
@@ -166,7 +135,6 @@ function main() {
|
||||
return;
|
||||
}
|
||||
|
||||
// Calculate stats
|
||||
const totalDuplicates = duplicateGroups.reduce((sum, g) => sum + g.delete_ids.length, 0);
|
||||
const affectedSessions = new Set(duplicateGroups.map(g => g.memory_session_id)).size;
|
||||
|
||||
@@ -178,7 +146,6 @@ function main() {
|
||||
console.log(`Observations after cleanup: ${totalCount.count - totalDuplicates}`);
|
||||
console.log('');
|
||||
|
||||
// Show sample of duplicates
|
||||
console.log('SAMPLE DUPLICATES (first 10 groups):');
|
||||
console.log('-'.repeat(60));
|
||||
|
||||
@@ -195,14 +162,12 @@ function main() {
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Execute deletion if not dry run
|
||||
if (!dryRun) {
|
||||
console.log('EXECUTING DELETION...');
|
||||
console.log('-'.repeat(60));
|
||||
|
||||
const allDeleteIds = duplicateGroups.flatMap(g => g.delete_ids);
|
||||
|
||||
// Delete in batches of 500 to avoid SQLite limits
|
||||
const BATCH_SIZE = 500;
|
||||
let deleted = 0;
|
||||
|
||||
@@ -222,7 +187,6 @@ function main() {
|
||||
console.log('');
|
||||
console.log(`Successfully deleted ${deleted} duplicate observations!`);
|
||||
|
||||
// Verify final count
|
||||
const finalCount = db.prepare('SELECT COUNT(*) as count FROM observations').get() as { count: number };
|
||||
console.log(`Final observation count: ${finalCount.count}`);
|
||||
|
||||
|
||||
+57
-178
@@ -1,98 +1,23 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* Clear messages from the queue
|
||||
*
|
||||
* Usage:
|
||||
* bun scripts/clear-failed-queue.ts # Clear failed messages (interactive)
|
||||
* bun scripts/clear-failed-queue.ts --all # Clear ALL messages (pending, processing, failed)
|
||||
* bun scripts/clear-failed-queue.ts --force # Non-interactive - clear without prompting
|
||||
*/
|
||||
|
||||
const WORKER_URL = 'http://localhost:37777';
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { existsSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
interface QueueMessage {
|
||||
id: number;
|
||||
session_db_id: number;
|
||||
message_type: string;
|
||||
tool_name: string | null;
|
||||
status: 'pending' | 'processing' | 'failed';
|
||||
retry_count: number;
|
||||
created_at_epoch: number;
|
||||
project: string | null;
|
||||
}
|
||||
interface CountRow { count: number }
|
||||
interface StatusRow { status: string; count: number }
|
||||
|
||||
interface QueueResponse {
|
||||
queue: {
|
||||
messages: QueueMessage[];
|
||||
totalPending: number;
|
||||
totalProcessing: number;
|
||||
totalFailed: number;
|
||||
stuckCount: number;
|
||||
};
|
||||
recentlyProcessed: QueueMessage[];
|
||||
sessionsWithPendingWork: number[];
|
||||
}
|
||||
|
||||
interface ClearResponse {
|
||||
success: boolean;
|
||||
clearedCount: number;
|
||||
}
|
||||
|
||||
async function checkWorkerHealth(): Promise<boolean> {
|
||||
try {
|
||||
const res = await fetch(`${WORKER_URL}/api/health`);
|
||||
return res.ok;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function getQueueStatus(): Promise<QueueResponse> {
|
||||
const res = await fetch(`${WORKER_URL}/api/pending-queue`);
|
||||
if (!res.ok) {
|
||||
throw new Error(`Failed to get queue status: ${res.status}`);
|
||||
}
|
||||
return res.json();
|
||||
}
|
||||
|
||||
async function clearFailedQueue(): Promise<ClearResponse> {
|
||||
const res = await fetch(`${WORKER_URL}/api/pending-queue/failed`, {
|
||||
method: 'DELETE'
|
||||
});
|
||||
if (!res.ok) {
|
||||
throw new Error(`Failed to clear failed queue: ${res.status}`);
|
||||
}
|
||||
return res.json();
|
||||
}
|
||||
|
||||
async function clearAllQueue(): Promise<ClearResponse> {
|
||||
const res = await fetch(`${WORKER_URL}/api/pending-queue/all`, {
|
||||
method: 'DELETE'
|
||||
});
|
||||
if (!res.ok) {
|
||||
throw new Error(`Failed to clear queue: ${res.status}`);
|
||||
}
|
||||
return res.json();
|
||||
}
|
||||
|
||||
function formatAge(epochMs: number): string {
|
||||
const ageMs = Date.now() - epochMs;
|
||||
const minutes = Math.floor(ageMs / 60000);
|
||||
const hours = Math.floor(minutes / 60);
|
||||
const days = Math.floor(hours / 24);
|
||||
|
||||
if (days > 0) return `${days}d ${hours % 24}h ago`;
|
||||
if (hours > 0) return `${hours}h ${minutes % 60}m ago`;
|
||||
return `${minutes}m ago`;
|
||||
function resolveDbPath(): string {
|
||||
const dataDir = process.env.CLAUDE_MEM_DATA_DIR || join(homedir(), '.claude-mem');
|
||||
return join(dataDir, 'claude-mem.db');
|
||||
}
|
||||
|
||||
async function prompt(question: string): Promise<string> {
|
||||
// Check if we have a TTY for interactive input
|
||||
if (!process.stdin.isTTY) {
|
||||
console.log(question + '(no TTY, use --force flag for non-interactive mode)');
|
||||
return 'n';
|
||||
}
|
||||
|
||||
return new Promise((resolve) => {
|
||||
process.stdout.write(question);
|
||||
process.stdin.setRawMode(false);
|
||||
@@ -107,40 +32,30 @@ async function prompt(question: string): Promise<string> {
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
|
||||
// Help flag
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
console.log(`
|
||||
Claude-Mem Queue Clearer
|
||||
|
||||
Clear messages from the observation queue.
|
||||
Clear orphaned messages from the pending_messages SQLite table.
|
||||
|
||||
Usage:
|
||||
bun scripts/clear-failed-queue.ts [options]
|
||||
|
||||
Options:
|
||||
--help, -h Show this help message
|
||||
--all Clear ALL messages (pending, processing, and failed)
|
||||
--all Clear ALL messages (pending, processing, processed, failed)
|
||||
--force Clear without prompting for confirmation
|
||||
|
||||
Examples:
|
||||
# Clear failed messages interactively
|
||||
bun scripts/clear-failed-queue.ts
|
||||
|
||||
# Clear ALL messages (pending, processing, failed)
|
||||
bun scripts/clear-failed-queue.ts --all
|
||||
|
||||
# Clear without confirmation (non-interactive)
|
||||
bun scripts/clear-failed-queue.ts --force
|
||||
|
||||
# Clear all messages without confirmation
|
||||
# Clear ALL messages without confirmation
|
||||
bun scripts/clear-failed-queue.ts --all --force
|
||||
|
||||
What is this for?
|
||||
Failed messages are observations that exceeded the maximum retry count.
|
||||
Processing/pending messages may be stuck or unwanted.
|
||||
This command removes them to clean up the queue.
|
||||
|
||||
--all is useful for a complete reset when you want to start fresh.
|
||||
Notes:
|
||||
Operates directly on ~/.claude-mem/claude-mem.db (or \$CLAUDE_MEM_DATA_DIR).
|
||||
Uses SQLite WAL mode so it is safe to run while the worker is running.
|
||||
`);
|
||||
process.exit(0);
|
||||
}
|
||||
@@ -152,102 +67,66 @@ What is this for?
|
||||
? '\n=== Claude-Mem Queue Clearer (ALL) ===\n'
|
||||
: '\n=== Claude-Mem Queue Clearer (Failed) ===\n');
|
||||
|
||||
// Check worker health
|
||||
const healthy = await checkWorkerHealth();
|
||||
if (!healthy) {
|
||||
console.log('Worker is not running. Start it with:');
|
||||
console.log(' cd ~/.claude/plugins/marketplaces/thedotmack && npm run worker:start\n');
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('Worker status: Running\n');
|
||||
|
||||
// Get queue status
|
||||
const status = await getQueueStatus();
|
||||
const { queue } = status;
|
||||
|
||||
console.log('Queue Summary:');
|
||||
console.log(` Pending: ${queue.totalPending}`);
|
||||
console.log(` Processing: ${queue.totalProcessing}`);
|
||||
console.log(` Failed: ${queue.totalFailed}`);
|
||||
console.log('');
|
||||
|
||||
// Check if there are messages to clear
|
||||
const totalToClear = clearAll
|
||||
? queue.totalPending + queue.totalProcessing + queue.totalFailed
|
||||
: queue.totalFailed;
|
||||
|
||||
if (totalToClear === 0) {
|
||||
console.log(clearAll
|
||||
? 'No messages in queue. Nothing to clear.\n'
|
||||
: 'No failed messages in queue. Nothing to clear.\n');
|
||||
const dbPath = resolveDbPath();
|
||||
if (!existsSync(dbPath)) {
|
||||
console.log(`No database found at ${dbPath}. Nothing to clear.\n`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Show details about messages to clear
|
||||
const messagesToShow = clearAll ? queue.messages : queue.messages.filter(m => m.status === 'failed');
|
||||
if (messagesToShow.length > 0) {
|
||||
console.log(clearAll ? 'Messages to Clear:' : 'Failed Messages:');
|
||||
console.log('─'.repeat(80));
|
||||
const db = new Database(dbPath);
|
||||
db.run('PRAGMA journal_mode = WAL');
|
||||
|
||||
// Group by session
|
||||
const bySession = new Map<number, QueueMessage[]>();
|
||||
for (const msg of messagesToShow) {
|
||||
const list = bySession.get(msg.session_db_id) || [];
|
||||
list.push(msg);
|
||||
bySession.set(msg.session_db_id, list);
|
||||
}
|
||||
const counts = db.prepare(
|
||||
'SELECT status, COUNT(*) as count FROM pending_messages GROUP BY status'
|
||||
).all() as StatusRow[];
|
||||
|
||||
for (const [sessionId, messages] of bySession) {
|
||||
const project = messages[0].project || 'unknown';
|
||||
const oldest = Math.min(...messages.map(m => m.created_at_epoch));
|
||||
const total = counts.reduce((sum, row) => sum + row.count, 0);
|
||||
const failed = counts.find(r => r.status === 'failed')?.count ?? 0;
|
||||
|
||||
if (clearAll) {
|
||||
const statuses = {
|
||||
pending: messages.filter(m => m.status === 'pending').length,
|
||||
processing: messages.filter(m => m.status === 'processing').length,
|
||||
failed: messages.filter(m => m.status === 'failed').length
|
||||
};
|
||||
console.log(` Session ${sessionId} (${project})`);
|
||||
console.log(` Messages: ${messages.length} total (${statuses.pending} pending, ${statuses.processing} processing, ${statuses.failed} failed)`);
|
||||
console.log(` Age: ${formatAge(oldest)}`);
|
||||
} else {
|
||||
console.log(` Session ${sessionId} (${project})`);
|
||||
console.log(` Messages: ${messages.length} failed`);
|
||||
console.log(` Age: ${formatAge(oldest)}`);
|
||||
}
|
||||
}
|
||||
console.log('─'.repeat(80));
|
||||
console.log('');
|
||||
console.log('Queue Summary:');
|
||||
for (const status of ['pending', 'processing', 'processed', 'failed'] as const) {
|
||||
const row = counts.find(r => r.status === status);
|
||||
console.log(` ${status.padEnd(11)} ${row?.count ?? 0}`);
|
||||
}
|
||||
console.log('');
|
||||
|
||||
const willClear = clearAll ? total : failed;
|
||||
if (willClear === 0) {
|
||||
console.log(clearAll
|
||||
? 'No messages in queue. Nothing to clear.\n'
|
||||
: 'No failed messages in queue. Nothing to clear.\n');
|
||||
db.close();
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Confirm before clearing
|
||||
const clearMessage = clearAll
|
||||
? `Clear ${totalToClear} messages (pending, processing, and failed)?`
|
||||
: `Clear ${queue.totalFailed} failed messages?`;
|
||||
|
||||
if (force) {
|
||||
console.log(`${clearMessage.replace('?', '')}...\n`);
|
||||
} else {
|
||||
const answer = await prompt(`${clearMessage} [y/N]: `);
|
||||
if (!force) {
|
||||
const answer = await prompt(
|
||||
clearAll
|
||||
? `Clear ${willClear} messages (all statuses)? [y/N]: `
|
||||
: `Clear ${willClear} failed messages? [y/N]: `
|
||||
);
|
||||
if (answer.toLowerCase() !== 'y') {
|
||||
console.log('\nCancelled. Run with --force to skip confirmation.\n');
|
||||
db.close();
|
||||
process.exit(0);
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Clear the queue
|
||||
const result = clearAll ? await clearAllQueue() : await clearFailedQueue();
|
||||
const stmt = clearAll
|
||||
? db.prepare('DELETE FROM pending_messages')
|
||||
: db.prepare("DELETE FROM pending_messages WHERE status = 'failed'");
|
||||
const cleared = stmt.run().changes;
|
||||
|
||||
const remaining = (db.prepare(
|
||||
'SELECT COUNT(*) as count FROM pending_messages'
|
||||
).get() as CountRow).count;
|
||||
|
||||
console.log('Clearing Result:');
|
||||
console.log(` Messages cleared: ${result.clearedCount}`);
|
||||
console.log(` Status: ${result.success ? 'Success' : 'Failed'}\n`);
|
||||
console.log(` Messages cleared: ${cleared}`);
|
||||
console.log(` Remaining: ${remaining}\n`);
|
||||
|
||||
if (result.success && result.clearedCount > 0) {
|
||||
console.log(clearAll
|
||||
? 'All messages have been removed from the queue.\n'
|
||||
: 'Failed messages have been removed from the queue.\n');
|
||||
}
|
||||
db.close();
|
||||
}
|
||||
|
||||
main().catch(err => {
|
||||
|
||||
@@ -1,20 +1,4 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* cwd-remap — Rewrite sdk_sessions.project (+ observations.project,
|
||||
* session_summaries.project) using the cwd captured per-message in
|
||||
* pending_messages.cwd as the single source of truth.
|
||||
*
|
||||
* For each distinct cwd:
|
||||
* - git -C <cwd> rev-parse --git-dir AND --git-common-dir
|
||||
* If they differ → worktree. parent = basename(dirname(common-dir)),
|
||||
* project = parent/<basename(cwd)>.
|
||||
* Else → project = basename(cwd).
|
||||
* - If the directory doesn't exist, or git errors, skip that cwd.
|
||||
*
|
||||
* Usage:
|
||||
* bun scripts/cwd-remap.ts # dry-run (default)
|
||||
* bun scripts/cwd-remap.ts --apply # write updates in a single transaction
|
||||
*/
|
||||
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { homedir } from 'os';
|
||||
@@ -51,7 +35,6 @@ function classify(cwd: string): Classification {
|
||||
const commonDir = git(cwd, ['rev-parse', '--path-format=absolute', '--git-common-dir']);
|
||||
if (!commonDir) return { kind: 'skip', reason: 'no-common-dir' };
|
||||
|
||||
// Use the worktree root, not the cwd — a session may be in a subdir.
|
||||
const toplevel = git(cwd, ['rev-parse', '--show-toplevel']);
|
||||
if (!toplevel) return { kind: 'skip', reason: 'no-toplevel' };
|
||||
const leaf = basename(toplevel);
|
||||
@@ -60,8 +43,6 @@ function classify(cwd: string): Classification {
|
||||
return { kind: 'main', project: leaf };
|
||||
}
|
||||
|
||||
// worktree: common-dir = <parent-repo>/.git (normal) or <parent>.git (bare).
|
||||
// Normal: dirname strips the trailing /.git. Bare: strip the .git suffix.
|
||||
const parentRepoDir = commonDir.endsWith('/.git')
|
||||
? dirname(commonDir)
|
||||
: commonDir.replace(/\.git$/, '');
|
||||
@@ -101,16 +82,12 @@ function main() {
|
||||
}
|
||||
console.log(` main=${counts.main} worktree=${counts.worktree} skip=${counts.skip}`);
|
||||
|
||||
// Skipped cwds (so user sees what's missing)
|
||||
const skipped = [...byCwd.entries()].filter(([, c]) => c.kind === 'skip') as Array<[string, Extract<Classification, { kind: 'skip' }>]>;
|
||||
if (skipped.length) {
|
||||
console.log('\nSkipped cwds:');
|
||||
for (const [cwd, c] of skipped) console.log(` [${c.reason}] ${cwd}`);
|
||||
}
|
||||
|
||||
// Per-session target: use the EARLIEST pending_messages.cwd for each session.
|
||||
// (Dominant-cwd is wrong: claude-mem's own hooks run from nested dirs like
|
||||
// `.context/claude-mem/` and dominate the count, misattributing the session.)
|
||||
const sessionRows = db.prepare(`
|
||||
SELECT s.id AS session_id, s.memory_session_id, s.content_session_id, s.project AS old_project, p.cwd
|
||||
FROM sdk_sessions s
|
||||
|
||||
@@ -1,113 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Debug Transcript Structure
|
||||
* Examines the first few entries to understand the conversation flow
|
||||
*/
|
||||
|
||||
import { TranscriptParser } from '../src/utils/transcript-parser.js';
|
||||
|
||||
const transcriptPath = process.argv[2];
|
||||
|
||||
if (!transcriptPath) {
|
||||
console.error('Usage: tsx scripts/debug-transcript-structure.ts <path-to-transcript.jsonl>');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const parser = new TranscriptParser(transcriptPath);
|
||||
const entries = parser.getAllEntries();
|
||||
|
||||
console.log(`Total entries: ${entries.length}\n`);
|
||||
|
||||
// Count entry types
|
||||
const typeCounts: Record<string, number> = {};
|
||||
for (const entry of entries) {
|
||||
typeCounts[entry.type] = (typeCounts[entry.type] || 0) + 1;
|
||||
}
|
||||
|
||||
console.log('Entry types:');
|
||||
for (const [type, count] of Object.entries(typeCounts)) {
|
||||
console.log(` ${type}: ${count}`);
|
||||
}
|
||||
|
||||
// Find first user and assistant entries
|
||||
const firstUser = entries.find(e => e.type === 'user');
|
||||
const firstAssistant = entries.find(e => e.type === 'assistant');
|
||||
|
||||
if (firstUser) {
|
||||
const userIndex = entries.indexOf(firstUser);
|
||||
console.log(`\n\n=== First User Entry (index ${userIndex}) ===`);
|
||||
console.log(`Timestamp: ${firstUser.timestamp}`);
|
||||
if (typeof firstUser.content === 'string') {
|
||||
console.log(`Content (string): ${firstUser.content.substring(0, 200)}...`);
|
||||
} else if (Array.isArray(firstUser.content)) {
|
||||
console.log(`Content blocks: ${firstUser.content.length}`);
|
||||
for (const block of firstUser.content) {
|
||||
if (block.type === 'text') {
|
||||
console.log(` - text: ${(block as any).text?.substring(0, 200)}...`);
|
||||
} else {
|
||||
console.log(` - ${block.type}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (firstAssistant) {
|
||||
const assistantIndex = entries.indexOf(firstAssistant);
|
||||
console.log(`\n\n=== First Assistant Entry (index ${assistantIndex}) ===`);
|
||||
console.log(`Timestamp: ${firstAssistant.timestamp}`);
|
||||
if (Array.isArray(firstAssistant.content)) {
|
||||
console.log(`Content blocks: ${firstAssistant.content.length}`);
|
||||
for (const block of firstAssistant.content) {
|
||||
if (block.type === 'text') {
|
||||
console.log(` - text: ${(block as any).text?.substring(0, 200)}...`);
|
||||
} else if (block.type === 'thinking') {
|
||||
console.log(` - thinking: ${(block as any).thinking?.substring(0, 200)}...`);
|
||||
} else if (block.type === 'tool_use') {
|
||||
console.log(` - tool_use: ${(block as any).name}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Find a few more user/assistant pairs
|
||||
console.log('\n\n=== First 3 Conversation Exchanges ===\n');
|
||||
|
||||
let userCount = 0;
|
||||
let assistantCount = 0;
|
||||
let exchangeNum = 0;
|
||||
|
||||
for (const entry of entries) {
|
||||
if (entry.type === 'user') {
|
||||
userCount++;
|
||||
if (userCount <= 3) {
|
||||
exchangeNum++;
|
||||
console.log(`\n--- Exchange ${exchangeNum}: USER ---`);
|
||||
if (typeof entry.content === 'string') {
|
||||
console.log(entry.content.substring(0, 150) + (entry.content.length > 150 ? '...' : ''));
|
||||
} else if (Array.isArray(entry.content)) {
|
||||
const textBlock = entry.content.find((b: any) => b.type === 'text');
|
||||
if (textBlock) {
|
||||
const text = (textBlock as any).text || '';
|
||||
console.log(text.substring(0, 150) + (text.length > 150 ? '...' : ''));
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if (entry.type === 'assistant' && userCount <= 3) {
|
||||
assistantCount++;
|
||||
if (Array.isArray(entry.content)) {
|
||||
const textBlock = entry.content.find((b: any) => b.type === 'text');
|
||||
const toolUses = entry.content.filter((b: any) => b.type === 'tool_use');
|
||||
|
||||
console.log(`\n--- Exchange ${exchangeNum}: ASSISTANT ---`);
|
||||
if (textBlock) {
|
||||
const text = (textBlock as any).text || '';
|
||||
console.log(text.substring(0, 150) + (text.length > 150 ? '...' : ''));
|
||||
}
|
||||
if (toolUses.length > 0) {
|
||||
console.log(`\nTools used: ${toolUses.map((t: any) => t.name).join(', ')}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (userCount >= 3 && assistantCount >= 3) break;
|
||||
}
|
||||
@@ -1,15 +1,5 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Post release notification to Discord
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/discord-release-notify.js v7.4.2
|
||||
* node scripts/discord-release-notify.js v7.4.2 "Custom release notes"
|
||||
*
|
||||
* Requires DISCORD_UPDATES_WEBHOOK in .env file
|
||||
*/
|
||||
|
||||
import { execSync } from 'child_process';
|
||||
import { readFileSync, existsSync } from 'fs';
|
||||
import { resolve, dirname } from 'path';
|
||||
@@ -49,7 +39,6 @@ function getReleaseNotes(version) {
|
||||
}
|
||||
|
||||
function cleanNotes(notes) {
|
||||
// Remove Claude Code footer and clean up
|
||||
return notes
|
||||
.replace(/🤖 Generated with \[Claude Code\].*$/s, '')
|
||||
.replace(/---\n*$/s, '')
|
||||
|
||||
@@ -1,99 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Simple 1:1 transcript dump in readable markdown format
|
||||
* Shows exactly what's in the transcript, chronologically
|
||||
*/
|
||||
|
||||
import { TranscriptParser } from '../src/utils/transcript-parser.js';
|
||||
import { writeFileSync } from 'fs';
|
||||
|
||||
const transcriptPath = process.argv[2];
|
||||
|
||||
if (!transcriptPath) {
|
||||
console.error('Usage: tsx scripts/dump-transcript-readable.ts <path-to-transcript.jsonl>');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const parser = new TranscriptParser(transcriptPath);
|
||||
const entries = parser.getAllEntries();
|
||||
|
||||
let output = '# Transcript Dump\n\n';
|
||||
output += `Total entries: ${entries.length}\n\n`;
|
||||
output += '---\n\n';
|
||||
|
||||
let entryNum = 0;
|
||||
|
||||
for (const entry of entries) {
|
||||
entryNum++;
|
||||
|
||||
// Skip file-history-snapshot and summary entries for now
|
||||
if (entry.type === 'file-history-snapshot' || entry.type === 'summary') continue;
|
||||
|
||||
output += `## Entry ${entryNum}: ${entry.type.toUpperCase()}\n`;
|
||||
output += `**Timestamp:** ${entry.timestamp}\n\n`;
|
||||
|
||||
if (entry.type === 'user') {
|
||||
const content = entry.message.content;
|
||||
|
||||
if (typeof content === 'string') {
|
||||
output += `**Content:**\n\`\`\`\n${content}\n\`\`\`\n\n`;
|
||||
} else if (Array.isArray(content)) {
|
||||
for (const block of content) {
|
||||
if (block.type === 'text') {
|
||||
output += `**Text:**\n\`\`\`\n${(block as any).text}\n\`\`\`\n\n`;
|
||||
} else if (block.type === 'tool_result') {
|
||||
output += `**Tool Result (${(block as any).tool_use_id}):**\n`;
|
||||
const resultContent = (block as any).content;
|
||||
if (typeof resultContent === 'string') {
|
||||
const preview = resultContent.substring(0, 500);
|
||||
output += `\`\`\`\n${preview}${resultContent.length > 500 ? '\n...(truncated)' : ''}\n\`\`\`\n\n`;
|
||||
} else {
|
||||
output += `\`\`\`json\n${JSON.stringify(resultContent, null, 2).substring(0, 500)}\n\`\`\`\n\n`;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (entry.type === 'assistant') {
|
||||
const content = entry.message.content;
|
||||
|
||||
if (Array.isArray(content)) {
|
||||
for (const block of content) {
|
||||
if (block.type === 'text') {
|
||||
output += `**Text:**\n\`\`\`\n${(block as any).text}\n\`\`\`\n\n`;
|
||||
} else if (block.type === 'thinking') {
|
||||
output += `**Thinking:**\n\`\`\`\n${(block as any).thinking}\n\`\`\`\n\n`;
|
||||
} else if (block.type === 'tool_use') {
|
||||
const tool = block as any;
|
||||
output += `**Tool Use: ${tool.name}**\n`;
|
||||
output += `\`\`\`json\n${JSON.stringify(tool.input, null, 2)}\n\`\`\`\n\n`;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Show token usage if available
|
||||
const usage = entry.message.usage;
|
||||
if (usage) {
|
||||
output += `**Usage:**\n`;
|
||||
output += `- Input: ${usage.input_tokens || 0}\n`;
|
||||
output += `- Output: ${usage.output_tokens || 0}\n`;
|
||||
output += `- Cache creation: ${usage.cache_creation_input_tokens || 0}\n`;
|
||||
output += `- Cache read: ${usage.cache_read_input_tokens || 0}\n\n`;
|
||||
}
|
||||
}
|
||||
|
||||
output += '---\n\n';
|
||||
|
||||
// Limit to first 20 entries to keep file manageable
|
||||
if (entryNum >= 20) {
|
||||
output += `\n_Remaining ${entries.length - 20} entries omitted for brevity_\n`;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
const outputPath = '/Users/alexnewman/Scripts/claude-mem/docs/context/transcript-dump.md';
|
||||
writeFileSync(outputPath, output, 'utf-8');
|
||||
|
||||
console.log(`\nTranscript dumped to: ${outputPath}`);
|
||||
console.log(`Showing first 20 conversation entries (skipped file-history-snapshot and summary types)\n`);
|
||||
@@ -1,11 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# E2E Test: Knowledge Agents
|
||||
# Fully hands-off test of the complete knowledge agent lifecycle.
|
||||
# Designed to be orchestrated via tmux-cli from Claude Code.
|
||||
#
|
||||
# Flow: health check → build corpus → list → get → prime → query → reprime → query → rebuild → delete → verify
|
||||
#
|
||||
set -euo pipefail
|
||||
|
||||
WORKER_URL="http://localhost:37777"
|
||||
@@ -14,8 +7,6 @@ PASS_COUNT=0
|
||||
FAIL_COUNT=0
|
||||
LOG_FILE="${HOME}/.claude-mem/logs/e2e-knowledge-agents-$(date +%Y%m%d-%H%M%S).log"
|
||||
|
||||
# -- Helpers ------------------------------------------------------------------
|
||||
|
||||
log() { echo "[$(date +%H:%M:%S)] $*" | tee -a "$LOG_FILE"; }
|
||||
pass() { PASS_COUNT=$((PASS_COUNT + 1)); log "PASS: $1"; }
|
||||
fail() { FAIL_COUNT=$((FAIL_COUNT + 1)); log "FAIL: $1 — $2"; }
|
||||
@@ -83,15 +74,11 @@ extract_body_and_status() {
|
||||
RESPONSE_STATUS=$(echo "$response" | tail -1)
|
||||
}
|
||||
|
||||
# -- Cleanup ------------------------------------------------------------------
|
||||
|
||||
cleanup_test_corpus() {
|
||||
log "Cleaning up test corpus '$CORPUS_NAME'..."
|
||||
curl -s -X DELETE "$WORKER_URL/api/corpus/$CORPUS_NAME" > /dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
# -- Tests --------------------------------------------------------------------
|
||||
|
||||
test_worker_health() {
|
||||
log "=== Test: Worker Health ==="
|
||||
local response
|
||||
@@ -132,7 +119,6 @@ test_list_corpora() {
|
||||
extract_body_and_status "$response"
|
||||
assert_http_status "List corpora" "200" "$RESPONSE_STATUS"
|
||||
|
||||
# Verify our test corpus is in the list
|
||||
local found
|
||||
found=$(echo "$RESPONSE_BODY" | jq -r ".[] | select(.name == \"$CORPUS_NAME\") | .name" 2>/dev/null)
|
||||
if [[ "$found" == "$CORPUS_NAME" ]]; then
|
||||
@@ -193,18 +179,15 @@ test_query_corpus() {
|
||||
|
||||
test_query_without_prime() {
|
||||
log "=== Test: Query Unprimed Corpus ==="
|
||||
# Build a second corpus but don't prime it
|
||||
curl_post "/api/corpus" "{\"name\": \"e2e-unprimed-test\", \"limit\": 5}" > /dev/null 2>&1
|
||||
local response
|
||||
response=$(curl_post "/api/corpus/e2e-unprimed-test/query" '{"question": "test"}' 30)
|
||||
extract_body_and_status "$response"
|
||||
# Should fail because corpus isn't primed
|
||||
if [[ "$RESPONSE_STATUS" != "200" ]] || echo "$RESPONSE_BODY" | jq -r '.error' 2>/dev/null | grep -qi "prime\|session"; then
|
||||
pass "Query unprimed corpus correctly rejected"
|
||||
else
|
||||
fail "Query unprimed corpus" "expected error about priming, got HTTP $RESPONSE_STATUS"
|
||||
fi
|
||||
# Cleanup
|
||||
curl -s -X DELETE "$WORKER_URL/api/corpus/e2e-unprimed-test" > /dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
@@ -212,7 +195,6 @@ test_reprime_corpus() {
|
||||
log "=== Test: Reprime Corpus ==="
|
||||
log " (Creating fresh session...)"
|
||||
|
||||
# Capture old session_id
|
||||
local old_response old_session_id
|
||||
old_response=$(curl_get "/api/corpus/$CORPUS_NAME")
|
||||
extract_body_and_status "$old_response"
|
||||
@@ -260,7 +242,6 @@ test_delete_corpus() {
|
||||
extract_body_and_status "$response"
|
||||
assert_http_status "Delete corpus" "200" "$RESPONSE_STATUS"
|
||||
|
||||
# Verify it's gone
|
||||
local verify_response
|
||||
verify_response=$(curl_get "/api/corpus/$CORPUS_NAME")
|
||||
extract_body_and_status "$verify_response"
|
||||
@@ -275,8 +256,6 @@ test_delete_nonexistent() {
|
||||
assert_http_status "Delete nonexistent returns 404" "404" "$RESPONSE_STATUS"
|
||||
}
|
||||
|
||||
# -- Main ---------------------------------------------------------------------
|
||||
|
||||
main() {
|
||||
mkdir -p "$(dirname "$LOG_FILE")"
|
||||
log "======================================================"
|
||||
@@ -285,39 +264,32 @@ main() {
|
||||
log "======================================================"
|
||||
log ""
|
||||
|
||||
# Cleanup any leftover test data
|
||||
cleanup_test_corpus
|
||||
|
||||
# Phase 1: Health checks
|
||||
test_worker_health
|
||||
test_worker_readiness
|
||||
log ""
|
||||
|
||||
# Phase 2: CRUD operations
|
||||
test_build_corpus
|
||||
test_list_corpora
|
||||
test_get_corpus
|
||||
test_get_corpus_404
|
||||
log ""
|
||||
|
||||
# Phase 3: Agent SDK operations (prime + query)
|
||||
test_prime_corpus
|
||||
test_query_corpus
|
||||
test_query_without_prime
|
||||
log ""
|
||||
|
||||
# Phase 4: Reprime + query again
|
||||
test_reprime_corpus
|
||||
test_query_after_reprime
|
||||
log ""
|
||||
|
||||
# Phase 5: Rebuild + cleanup
|
||||
test_rebuild_corpus
|
||||
test_delete_corpus
|
||||
test_delete_nonexistent
|
||||
log ""
|
||||
|
||||
# Summary
|
||||
local total=$((PASS_COUNT + FAIL_COUNT))
|
||||
log "======================================================"
|
||||
log " RESULTS: $PASS_COUNT/$total passed, $FAIL_COUNT failed"
|
||||
|
||||
@@ -1,273 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Endless Mode Token Economics Calculator
|
||||
*
|
||||
* Simulates the recursive/cumulative token savings from Endless Mode by
|
||||
* "playing the tape through" with real observation data from SQLite.
|
||||
*
|
||||
* Key Insight:
|
||||
* - Discovery tokens are ALWAYS spent (creating observations)
|
||||
* - But Endless Mode feeds compressed observations as context instead of full tool outputs
|
||||
* - Savings compound recursively - each tool benefits from ALL previous compressions
|
||||
*/
|
||||
|
||||
const observationsData = [{"id":10136,"type":"decision","title":"Token Accounting Function for Recursive Continuation Pattern","discovery_tokens":4037,"created_at_epoch":1763360747429,"compressed_size":1613},
|
||||
{"id":10135,"type":"discovery","title":"Sequential Thinking Analysis of Token Economics Calculator","discovery_tokens":1439,"created_at_epoch":1763360651617,"compressed_size":1812},
|
||||
{"id":10134,"type":"discovery","title":"Recent Context Query Execution","discovery_tokens":1273,"created_at_epoch":1763360646273,"compressed_size":1228},
|
||||
{"id":10133,"type":"discovery","title":"Token Data Query Execution and Historical Context","discovery_tokens":11878,"created_at_epoch":1763360642485,"compressed_size":1924},
|
||||
{"id":10132,"type":"discovery","title":"Token Data Query and Script Validation Request","discovery_tokens":4167,"created_at_epoch":1763360628269,"compressed_size":903},
|
||||
{"id":10131,"type":"discovery","title":"Endless Mode Token Economics Analysis Output: Complete Infrastructure Impact","discovery_tokens":2458,"created_at_epoch":1763360553238,"compressed_size":2166},
|
||||
{"id":10130,"type":"change","title":"Integration of Actual Compute Savings Analysis into Main Execution Flow","discovery_tokens":11031,"created_at_epoch":1763360545347,"compressed_size":1032},
|
||||
{"id":10129,"type":"discovery","title":"Prompt Caching Economics: User Cost vs. Anthropic Compute Cost Divergence","discovery_tokens":20059,"created_at_epoch":1763360540854,"compressed_size":1802},
|
||||
{"id":10128,"type":"discovery","title":"Token Caching Cost Analysis Across AI Model Providers","discovery_tokens":3506,"created_at_epoch":1763360478133,"compressed_size":1245},
|
||||
{"id":10127,"type":"discovery","title":"Endless Mode Token Economics Calculator Successfully Integrated Prompt Caching Cost Model","discovery_tokens":3481,"created_at_epoch":1763360384055,"compressed_size":2444},
|
||||
{"id":10126,"type":"bugfix","title":"Fix Return Statement Variable Names in playTheTapeThrough Function","discovery_tokens":8326,"created_at_epoch":1763360374566,"compressed_size":1250},
|
||||
{"id":10125,"type":"change","title":"Redesign Timeline Display to Show Fresh/Cached Token Breakdown and Real Dollar Costs","discovery_tokens":12999,"created_at_epoch":1763360368843,"compressed_size":2004},
|
||||
{"id":10124,"type":"change","title":"Replace Estimated Cost Model with Actual Caching-Based Costs in Anthropic Scale Analysis","discovery_tokens":12867,"created_at_epoch":1763360361147,"compressed_size":2064},
|
||||
{"id":10123,"type":"change","title":"Pivot Session Length Comparison Table from Token to Cost Metrics","discovery_tokens":9746,"created_at_epoch":1763360352992,"compressed_size":1652},
|
||||
{"id":10122,"type":"change","title":"Add Dual Reporting: Token Count vs Actual Cost in Comparison Output","discovery_tokens":9602,"created_at_epoch":1763360346495,"compressed_size":1640},
|
||||
{"id":10121,"type":"change","title":"Apply Prompt Caching Cost Model to Endless Mode Calculation Function","discovery_tokens":9963,"created_at_epoch":1763360339238,"compressed_size":2003},
|
||||
{"id":10120,"type":"change","title":"Integrate Prompt Caching Cost Calculations into Without-Endless-Mode Function","discovery_tokens":8652,"created_at_epoch":1763360332046,"compressed_size":1701},
|
||||
{"id":10119,"type":"change","title":"Display Prompt Caching Pricing in Initial Calculator Output","discovery_tokens":6669,"created_at_epoch":1763360325882,"compressed_size":1188},
|
||||
{"id":10118,"type":"change","title":"Add Prompt Caching Pricing Model to Token Economics Calculator","discovery_tokens":10433,"created_at_epoch":1763360320552,"compressed_size":1264},
|
||||
{"id":10117,"type":"discovery","title":"Claude API Prompt Caching Cost Optimization Factor","discovery_tokens":3439,"created_at_epoch":1763360210175,"compressed_size":1142},
|
||||
{"id":10116,"type":"discovery","title":"Endless Mode Token Economics Verified at Scale","discovery_tokens":2855,"created_at_epoch":1763360144039,"compressed_size":2184},
|
||||
{"id":10115,"type":"feature","title":"Token Economics Calculator for Endless Mode Sessions","discovery_tokens":13468,"created_at_epoch":1763360134068,"compressed_size":1858},
|
||||
{"id":10114,"type":"decision","title":"Token Accounting for Recursive Session Continuations","discovery_tokens":3550,"created_at_epoch":1763360052317,"compressed_size":1478},
|
||||
{"id":10113,"type":"discovery","title":"Performance and Token Optimization Impact Analysis for Endless Mode","discovery_tokens":3464,"created_at_epoch":1763359862175,"compressed_size":1259},
|
||||
{"id":10112,"type":"change","title":"Endless Mode Blocking Hooks & Transcript Transformation Plan Document Created","discovery_tokens":17312,"created_at_epoch":1763359465307,"compressed_size":2181},
|
||||
{"id":10111,"type":"change","title":"Plan Document Creation for Morning Implementation","discovery_tokens":3652,"created_at_epoch":1763359347166,"compressed_size":843},
|
||||
{"id":10110,"type":"decision","title":"Blocking vs Non-Blocking Behavior by Mode","discovery_tokens":3652,"created_at_epoch":1763359347165,"compressed_size":797},
|
||||
{"id":10109,"type":"decision","title":"Tool Use and Observation Processing Architecture: Non-Blocking vs Blocking","discovery_tokens":3472,"created_at_epoch":1763359247045,"compressed_size":1349},
|
||||
{"id":10108,"type":"feature","title":"SessionManager.getMessageIterator implements event-driven async generator with graceful abort handling","discovery_tokens":2417,"created_at_epoch":1763359189299,"compressed_size":2016},
|
||||
{"id":10107,"type":"feature","title":"SessionManager implements event-driven session lifecycle with auto-initialization and zero-latency queue notifications","discovery_tokens":4734,"created_at_epoch":1763359165608,"compressed_size":2781},
|
||||
{"id":10106,"type":"discovery","title":"Two distinct uses of transcript data: live data flow vs session initialization","discovery_tokens":2933,"created_at_epoch":1763359156448,"compressed_size":2015},
|
||||
{"id":10105,"type":"discovery","title":"Transcript initialization pattern identified for compressed context on session resume","discovery_tokens":2933,"created_at_epoch":1763359156447,"compressed_size":2536},
|
||||
{"id":10104,"type":"feature","title":"SDKAgent implements event-driven message generator with continuation prompt logic and Endless Mode integration","discovery_tokens":6148,"created_at_epoch":1763359140399,"compressed_size":3241},
|
||||
{"id":10103,"type":"discovery","title":"Endless Mode architecture documented with phased implementation plan and context economics","discovery_tokens":5296,"created_at_epoch":1763359127954,"compressed_size":3145},
|
||||
{"id":10102,"type":"feature","title":"Save hook enhanced to extract and forward tool_use_id for Endless Mode linking","discovery_tokens":3294,"created_at_epoch":1763359115848,"compressed_size":2125},
|
||||
{"id":10101,"type":"feature","title":"TransformLayer implements Endless Mode context compression via observation substitution","discovery_tokens":4637,"created_at_epoch":1763359108317,"compressed_size":2629},
|
||||
{"id":10100,"type":"feature","title":"EndlessModeConfig implemented for loading Endless Mode settings from files and environment","discovery_tokens":2313,"created_at_epoch":1763359099972,"compressed_size":2125},
|
||||
{"id":10098,"type":"change","title":"User prompts wrapped with semantic XML structure in buildInitPrompt and buildContinuationPrompt","discovery_tokens":7806,"created_at_epoch":1763359091460,"compressed_size":1585},
|
||||
{"id":10099,"type":"discovery","title":"Session persistence mechanism relies on SDK internal state without context reload","discovery_tokens":7806,"created_at_epoch":1763359091460,"compressed_size":1883},
|
||||
{"id":10097,"type":"change","title":"Worker service session init now extracts userPrompt and promptNumber from request body","discovery_tokens":7806,"created_at_epoch":1763359091459,"compressed_size":1148},
|
||||
{"id":10096,"type":"feature","title":"SessionManager enhanced to accept dynamic userPrompt updates during multi-turn conversations","discovery_tokens":7806,"created_at_epoch":1763359091457,"compressed_size":1528},
|
||||
{"id":10095,"type":"discovery","title":"Five lifecycle hooks integrate claude-mem at critical session boundaries","discovery_tokens":6625,"created_at_epoch":1763359074808,"compressed_size":1570},
|
||||
{"id":10094,"type":"discovery","title":"PostToolUse hook is real-time observation creation point, not delayed processing","discovery_tokens":6625,"created_at_epoch":1763359074807,"compressed_size":2371},
|
||||
{"id":10093,"type":"discovery","title":"PostToolUse hook timing and compression integration options explored","discovery_tokens":1696,"created_at_epoch":1763359062088,"compressed_size":1605},
|
||||
{"id":10092,"type":"discovery","title":"Transcript transformation strategy for endless mode identified","discovery_tokens":6112,"created_at_epoch":1763359057563,"compressed_size":1968},
|
||||
{"id":10091,"type":"decision","title":"Finalized Transcript Compression Implementation Strategy","discovery_tokens":1419,"created_at_epoch":1763358943803,"compressed_size":1556},
|
||||
{"id":10090,"type":"discovery","title":"UserPromptSubmit Hook as Compression Integration Point","discovery_tokens":1546,"created_at_epoch":1763358931936,"compressed_size":1621},
|
||||
{"id":10089,"type":"decision","title":"Hypothesis 5 Selected: UserPromptSubmit Hook for Transcript Compression","discovery_tokens":1465,"created_at_epoch":1763358920209,"compressed_size":1918}];
|
||||
|
||||
// Estimate original tool output size from discovery tokens
|
||||
// Heuristic: discovery_tokens roughly correlates with original content size
|
||||
// Assumption: If it took 10k tokens to analyze, original was probably 15-30k tokens
|
||||
function estimateOriginalToolOutputSize(discoveryTokens) {
|
||||
// Conservative multiplier: 2x (original content was 2x the discovery cost)
|
||||
// This accounts for: reading the tool output + analyzing it + generating observation
|
||||
return discoveryTokens * 2;
|
||||
}
|
||||
|
||||
// Convert compressed_size (character count) to approximate token count
|
||||
// Rough heuristic: 1 token ≈ 4 characters for English text
|
||||
function charsToTokens(chars) {
|
||||
return Math.ceil(chars / 4);
|
||||
}
|
||||
|
||||
/**
|
||||
* Simulate session WITHOUT Endless Mode (current behavior)
|
||||
* Each continuation carries ALL previous full tool outputs in context
|
||||
*/
|
||||
function calculateWithoutEndlessMode(observations) {
|
||||
let cumulativeContextTokens = 0;
|
||||
let totalDiscoveryTokens = 0;
|
||||
let totalContinuationTokens = 0;
|
||||
const timeline = [];
|
||||
|
||||
observations.forEach((obs, index) => {
|
||||
const toolNumber = index + 1;
|
||||
const originalToolSize = estimateOriginalToolOutputSize(obs.discovery_tokens);
|
||||
|
||||
// Discovery cost (creating observation from full tool output)
|
||||
const discoveryCost = obs.discovery_tokens;
|
||||
totalDiscoveryTokens += discoveryCost;
|
||||
|
||||
// Continuation cost: Re-process ALL previous tool outputs + current one
|
||||
// This is the key recursive cost
|
||||
cumulativeContextTokens += originalToolSize;
|
||||
const continuationCost = cumulativeContextTokens;
|
||||
totalContinuationTokens += continuationCost;
|
||||
|
||||
timeline.push({
|
||||
tool: toolNumber,
|
||||
obsId: obs.id,
|
||||
title: obs.title.substring(0, 60),
|
||||
originalSize: originalToolSize,
|
||||
discoveryCost,
|
||||
contextSize: cumulativeContextTokens,
|
||||
continuationCost,
|
||||
totalCostSoFar: totalDiscoveryTokens + totalContinuationTokens
|
||||
});
|
||||
});
|
||||
|
||||
return {
|
||||
totalDiscoveryTokens,
|
||||
totalContinuationTokens,
|
||||
totalTokens: totalDiscoveryTokens + totalContinuationTokens,
|
||||
timeline
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Simulate session WITH Endless Mode
|
||||
* Each continuation carries ALL previous COMPRESSED observations in context
|
||||
*/
|
||||
function calculateWithEndlessMode(observations) {
|
||||
let cumulativeContextTokens = 0;
|
||||
let totalDiscoveryTokens = 0;
|
||||
let totalContinuationTokens = 0;
|
||||
const timeline = [];
|
||||
|
||||
observations.forEach((obs, index) => {
|
||||
const toolNumber = index + 1;
|
||||
const originalToolSize = estimateOriginalToolOutputSize(obs.discovery_tokens);
|
||||
const compressedSize = charsToTokens(obs.compressed_size);
|
||||
|
||||
// Discovery cost (same as without Endless Mode - still need to create observation)
|
||||
const discoveryCost = obs.discovery_tokens;
|
||||
totalDiscoveryTokens += discoveryCost;
|
||||
|
||||
// KEY DIFFERENCE: Add COMPRESSED size to context, not original size
|
||||
cumulativeContextTokens += compressedSize;
|
||||
const continuationCost = cumulativeContextTokens;
|
||||
totalContinuationTokens += continuationCost;
|
||||
|
||||
const compressionRatio = ((originalToolSize - compressedSize) / originalToolSize * 100).toFixed(1);
|
||||
|
||||
timeline.push({
|
||||
tool: toolNumber,
|
||||
obsId: obs.id,
|
||||
title: obs.title.substring(0, 60),
|
||||
originalSize: originalToolSize,
|
||||
compressedSize,
|
||||
compressionRatio: `${compressionRatio}%`,
|
||||
discoveryCost,
|
||||
contextSize: cumulativeContextTokens,
|
||||
continuationCost,
|
||||
totalCostSoFar: totalDiscoveryTokens + totalContinuationTokens
|
||||
});
|
||||
});
|
||||
|
||||
return {
|
||||
totalDiscoveryTokens,
|
||||
totalContinuationTokens,
|
||||
totalTokens: totalDiscoveryTokens + totalContinuationTokens,
|
||||
timeline
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Play the tape through - show token-by-token progression
|
||||
*/
|
||||
function playTheTapeThrough(observations) {
|
||||
console.log('\n' + '='.repeat(100));
|
||||
console.log('ENDLESS MODE TOKEN ECONOMICS CALCULATOR');
|
||||
console.log('Playing the tape through with REAL observation data');
|
||||
console.log('='.repeat(100) + '\n');
|
||||
|
||||
console.log(`📊 Dataset: ${observations.length} observations from live sessions\n`);
|
||||
|
||||
// Calculate both scenarios
|
||||
const without = calculateWithoutEndlessMode(observations);
|
||||
const withMode = calculateWithEndlessMode(observations);
|
||||
|
||||
// Show first 10 tools from each scenario side by side
|
||||
console.log('🎬 TAPE PLAYBACK: First 10 Tools\n');
|
||||
console.log('WITHOUT Endless Mode (Current) | WITH Endless Mode (Proposed)');
|
||||
console.log('-'.repeat(100));
|
||||
|
||||
for (let i = 0; i < Math.min(10, observations.length); i++) {
|
||||
const w = without.timeline[i];
|
||||
const e = withMode.timeline[i];
|
||||
|
||||
console.log(`\nTool #${w.tool}: ${w.title}`);
|
||||
console.log(` Original: ${w.originalSize.toLocaleString()}t | Compressed: ${e.compressedSize.toLocaleString()}t (${e.compressionRatio} saved)`);
|
||||
console.log(` Context: ${w.contextSize.toLocaleString()}t | Context: ${e.contextSize.toLocaleString()}t`);
|
||||
console.log(` Total: ${w.totalCostSoFar.toLocaleString()}t | Total: ${e.totalCostSoFar.toLocaleString()}t`);
|
||||
}
|
||||
|
||||
// Summary table
|
||||
console.log('\n' + '='.repeat(100));
|
||||
console.log('📈 FINAL TOTALS\n');
|
||||
|
||||
console.log('WITHOUT Endless Mode (Current):');
|
||||
console.log(` Discovery tokens: ${without.totalDiscoveryTokens.toLocaleString()}t (creating observations)`);
|
||||
console.log(` Continuation tokens: ${without.totalContinuationTokens.toLocaleString()}t (context accumulation)`);
|
||||
console.log(` TOTAL TOKENS: ${without.totalTokens.toLocaleString()}t`);
|
||||
|
||||
console.log('\nWITH Endless Mode:');
|
||||
console.log(` Discovery tokens: ${withMode.totalDiscoveryTokens.toLocaleString()}t (same - still create observations)`);
|
||||
console.log(` Continuation tokens: ${withMode.totalContinuationTokens.toLocaleString()}t (COMPRESSED context)`);
|
||||
console.log(` TOTAL TOKENS: ${withMode.totalTokens.toLocaleString()}t`);
|
||||
|
||||
const tokensSaved = without.totalTokens - withMode.totalTokens;
|
||||
const percentSaved = (tokensSaved / without.totalTokens * 100).toFixed(1);
|
||||
|
||||
console.log('\n💰 SAVINGS:');
|
||||
console.log(` Tokens saved: ${tokensSaved.toLocaleString()}t`);
|
||||
console.log(` Percentage saved: ${percentSaved}%`);
|
||||
console.log(` Efficiency gain: ${(without.totalTokens / withMode.totalTokens).toFixed(2)}x`);
|
||||
|
||||
// Anthropic scale calculation
|
||||
console.log('\n' + '='.repeat(100));
|
||||
console.log('🌍 ANTHROPIC SCALE IMPACT\n');
|
||||
|
||||
// Conservative assumptions
|
||||
const activeUsers = 100000; // Claude Code users
|
||||
const sessionsPerWeek = 10; // Per user
|
||||
const toolsPerSession = observations.length; // Use our actual data
|
||||
const weeklyToolUses = activeUsers * sessionsPerWeek * toolsPerSession;
|
||||
|
||||
const avgTokensPerToolWithout = without.totalTokens / observations.length;
|
||||
const avgTokensPerToolWith = withMode.totalTokens / observations.length;
|
||||
|
||||
const weeklyTokensWithout = weeklyToolUses * avgTokensPerToolWithout;
|
||||
const weeklyTokensWith = weeklyToolUses * avgTokensPerToolWith;
|
||||
const weeklyTokensSaved = weeklyTokensWithout - weeklyTokensWith;
|
||||
|
||||
console.log('Assumptions:');
|
||||
console.log(` Active Claude Code users: ${activeUsers.toLocaleString()}`);
|
||||
console.log(` Sessions per user/week: ${sessionsPerWeek}`);
|
||||
console.log(` Tools per session: ${toolsPerSession}`);
|
||||
console.log(` Weekly tool uses: ${weeklyToolUses.toLocaleString()}`);
|
||||
|
||||
console.log('\nWeekly Compute:');
|
||||
console.log(` Without Endless Mode: ${(weeklyTokensWithout / 1e9).toFixed(2)} billion tokens`);
|
||||
console.log(` With Endless Mode: ${(weeklyTokensWith / 1e9).toFixed(2)} billion tokens`);
|
||||
console.log(` Weekly savings: ${(weeklyTokensSaved / 1e9).toFixed(2)} billion tokens (${percentSaved}%)`);
|
||||
|
||||
const annualTokensSaved = weeklyTokensSaved * 52;
|
||||
console.log(` Annual savings: ${(annualTokensSaved / 1e12).toFixed(2)} TRILLION tokens`);
|
||||
|
||||
console.log('\n💡 What this means:');
|
||||
console.log(` • ${percentSaved}% reduction in Claude Code inference costs`);
|
||||
console.log(` • ${(without.totalTokens / withMode.totalTokens).toFixed(1)}x more users served with same infrastructure`);
|
||||
console.log(` • Massive energy/compute savings at scale`);
|
||||
console.log(` • Longer sessions = better UX without economic penalty`);
|
||||
|
||||
console.log('\n' + '='.repeat(100) + '\n');
|
||||
|
||||
return {
|
||||
without,
|
||||
withMode,
|
||||
tokensSaved,
|
||||
percentSaved,
|
||||
weeklyTokensSaved,
|
||||
annualTokensSaved
|
||||
};
|
||||
}
|
||||
|
||||
// Run the calculation
|
||||
playTheTapeThrough(observationsData);
|
||||
@@ -1,9 +1,4 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Export memories matching a search query to a portable JSON format
|
||||
* Usage: npx tsx scripts/export-memories.ts <query> <output-file> [--project=name]
|
||||
* Example: npx tsx scripts/export-memories.ts "windows" windows-memories.json --project=claude-mem
|
||||
*/
|
||||
|
||||
import { writeFileSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
@@ -19,14 +14,12 @@ import type {
|
||||
|
||||
async function exportMemories(query: string, outputFile: string, project?: string) {
|
||||
try {
|
||||
// Read port from settings
|
||||
const settings = SettingsDefaultsManager.loadFromFile(join(homedir(), '.claude-mem', 'settings.json'));
|
||||
const port = parseInt(settings.CLAUDE_MEM_WORKER_PORT, 10);
|
||||
const baseUrl = `http://localhost:${port}`;
|
||||
|
||||
console.log(`🔍 Searching for: "${query}"${project ? ` (project: ${project})` : ' (all projects)'}`);
|
||||
|
||||
// Build query params - use format=json for raw data
|
||||
const params = new URLSearchParams({
|
||||
query,
|
||||
format: 'json',
|
||||
@@ -34,7 +27,6 @@ async function exportMemories(query: string, outputFile: string, project?: strin
|
||||
});
|
||||
if (project) params.set('project', project);
|
||||
|
||||
// Unified search - gets all result types using hybrid search
|
||||
console.log('📡 Fetching all memories via hybrid search...');
|
||||
const searchResponse = await fetch(`${baseUrl}/api/search?${params.toString()}`);
|
||||
if (!searchResponse.ok) {
|
||||
@@ -50,7 +42,6 @@ async function exportMemories(query: string, outputFile: string, project?: strin
|
||||
console.log(`✅ Found ${summaries.length} session summaries`);
|
||||
console.log(`✅ Found ${prompts.length} user prompts`);
|
||||
|
||||
// Get unique memory session IDs from observations and summaries
|
||||
const memorySessionIds = new Set<string>();
|
||||
observations.forEach((o) => {
|
||||
if (o.memory_session_id) memorySessionIds.add(o.memory_session_id);
|
||||
@@ -59,7 +50,6 @@ async function exportMemories(query: string, outputFile: string, project?: strin
|
||||
if (s.memory_session_id) memorySessionIds.add(s.memory_session_id);
|
||||
});
|
||||
|
||||
// Get SDK sessions metadata via API
|
||||
console.log('📡 Fetching SDK sessions metadata...');
|
||||
let sessions: SdkSessionRecord[] = [];
|
||||
if (memorySessionIds.size > 0) {
|
||||
@@ -76,7 +66,6 @@ async function exportMemories(query: string, outputFile: string, project?: strin
|
||||
}
|
||||
console.log(`✅ Found ${sessions.length} SDK sessions`);
|
||||
|
||||
// Create export data
|
||||
const exportData: ExportData = {
|
||||
exportedAt: new Date().toISOString(),
|
||||
exportedAtEpoch: Date.now(),
|
||||
@@ -92,7 +81,6 @@ async function exportMemories(query: string, outputFile: string, project?: strin
|
||||
prompts
|
||||
};
|
||||
|
||||
// Write to file
|
||||
writeFileSync(outputFile, JSON.stringify(exportData, null, 2));
|
||||
|
||||
console.log(`\n📦 Export complete!`);
|
||||
@@ -109,7 +97,6 @@ async function exportMemories(query: string, outputFile: string, project?: strin
|
||||
}
|
||||
}
|
||||
|
||||
// CLI interface
|
||||
const args = process.argv.slice(2);
|
||||
if (args.length < 2) {
|
||||
console.error('Usage: npx tsx scripts/export-memories.ts <query> <output-file> [--project=name]');
|
||||
@@ -118,7 +105,6 @@ if (args.length < 2) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Parse arguments
|
||||
const [query, outputFile, ...flags] = args;
|
||||
const project = flags.find(f => f.startsWith('--project='))?.split('=')[1];
|
||||
|
||||
|
||||
@@ -1,178 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Extract prompt sections from src/sdk/prompts.ts and generate modes/code.yaml
|
||||
* This ensures the YAML contains the exact same wording as the hardcoded prompts
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
// Read the prompts.ts from main branch (saved to /tmp)
|
||||
const promptsPath = '/tmp/prompts-main.ts';
|
||||
const promptsContent = fs.readFileSync(promptsPath, 'utf-8');
|
||||
|
||||
// Extract buildInitPrompt function content
|
||||
const initPromptMatch = promptsContent.match(/export function buildInitPrompt\([^)]+\): string \{[\s\S]*?return `([\s\S]*?)`;\s*\}/);
|
||||
if (!initPromptMatch) {
|
||||
console.error('Could not find buildInitPrompt function');
|
||||
process.exit(1);
|
||||
}
|
||||
const initPrompt = initPromptMatch[1];
|
||||
|
||||
// Extract sections from buildInitPrompt
|
||||
// Line 41: observer_role starts with "Your job is to monitor..."
|
||||
const observerRoleMatch = initPrompt.match(/Your job is to monitor[^\n]*\n\n(?:SPATIAL AWARENESS:[\s\S]*?\n\n)?/);
|
||||
const observerRole = observerRoleMatch ? observerRoleMatch[0].replace(/\n\n$/, '') : '';
|
||||
|
||||
// Extract recording_focus (WHAT TO RECORD section)
|
||||
const recordingFocusMatch = initPrompt.match(/WHAT TO RECORD\n-{14}\n([\s\S]*?)(?=\n\nWHEN TO SKIP)/);
|
||||
const recordingFocus = recordingFocusMatch ? `WHAT TO RECORD\n--------------\n${recordingFocusMatch[1]}` : '';
|
||||
|
||||
// Extract skip_guidance (WHEN TO SKIP section)
|
||||
const skipGuidanceMatch = initPrompt.match(/WHEN TO SKIP\n-{12}\n([\s\S]*?)(?=\n\nOUTPUT FORMAT)/);
|
||||
const skipGuidance = skipGuidanceMatch ? `WHEN TO SKIP\n------------\n${skipGuidanceMatch[1]}` : '';
|
||||
|
||||
// Extract type_guidance (from XML comment)
|
||||
const typeGuidanceMatch = initPrompt.match(/<!--\n\s+\*\*type\*\*: MUST be EXACTLY[^\n]*\n([\s\S]*?)-->/);
|
||||
const typeGuidance = typeGuidanceMatch ? typeGuidanceMatch[0].replace(/<!--\n\s+/, '').replace(/\s+-->/, '').trim() : '';
|
||||
|
||||
// Extract field_guidance (facts AND files comments combined)
|
||||
const factsMatch = initPrompt.match(/\*\*facts\*\*: Concise[^\n]*\n([\s\S]*?)(?=\n -->)/);
|
||||
const filesMatch = initPrompt.match(/\*\*files\*\*:[^\n]*\n/);
|
||||
|
||||
const factsText = factsMatch ? `**facts**: Concise, self-contained statements\n${factsMatch[1].trim()}` : '';
|
||||
const filesText = filesMatch ? filesMatch[0].trim() : '**files**: All files touched (full paths from project root)';
|
||||
|
||||
const fieldGuidance = `${factsText}\n\n${filesText}`;
|
||||
|
||||
// Extract concept_guidance (concepts comment)
|
||||
const conceptGuidanceMatch = initPrompt.match(/<!--\n\s+\*\*concepts\*\*: 2-5 knowledge[^\n]*\n([\s\S]*?)-->/);
|
||||
const conceptGuidance = conceptGuidanceMatch ? conceptGuidanceMatch[0].replace(/<!--\n\s+/, '').replace(/\s+-->/, '').trim() : '';
|
||||
|
||||
// Build the JSON content
|
||||
const jsonData = {
|
||||
name: "Code Development",
|
||||
description: "Software development and engineering work",
|
||||
version: "1.0.0",
|
||||
observation_types: [
|
||||
{ id: "bugfix", label: "Bug Fix", description: "Something was broken, now fixed", emoji: "🔴", work_emoji: "🛠️" },
|
||||
{ id: "feature", label: "Feature", description: "New capability or functionality added", emoji: "🟣", work_emoji: "🛠️" },
|
||||
{ id: "refactor", label: "Refactor", description: "Code restructured, behavior unchanged", emoji: "🔄", work_emoji: "🛠️" },
|
||||
{ id: "change", label: "Change", description: "Generic modification (docs, config, misc)", emoji: "✅", work_emoji: "🛠️" },
|
||||
{ id: "discovery", label: "Discovery", description: "Learning about existing system", emoji: "🔵", work_emoji: "🔍" },
|
||||
{ id: "decision", label: "Decision", description: "Architectural/design choice with rationale", emoji: "⚖️", work_emoji: "⚖️" }
|
||||
],
|
||||
observation_concepts: [
|
||||
{ id: "how-it-works", label: "How It Works", description: "Understanding mechanisms" },
|
||||
{ id: "why-it-exists", label: "Why It Exists", description: "Purpose or rationale" },
|
||||
{ id: "what-changed", label: "What Changed", description: "Modifications made" },
|
||||
{ id: "problem-solution", label: "Problem-Solution", description: "Issues and their fixes" },
|
||||
{ id: "gotcha", label: "Gotcha", description: "Traps or edge cases" },
|
||||
{ id: "pattern", label: "Pattern", description: "Reusable approach" },
|
||||
{ id: "trade-off", label: "Trade-Off", description: "Pros/cons of a decision" }
|
||||
],
|
||||
prompts: {
|
||||
observer_role: observerRole,
|
||||
recording_focus: recordingFocus,
|
||||
skip_guidance: skipGuidance,
|
||||
type_guidance: typeGuidance,
|
||||
concept_guidance: conceptGuidance,
|
||||
field_guidance: fieldGuidance,
|
||||
format_examples: ""
|
||||
}
|
||||
};
|
||||
|
||||
// OLD YAML BUILD:
|
||||
const yamlContent_OLD = `name: "Code Development"
|
||||
description: "Software development and engineering work"
|
||||
version: "1.0.0"
|
||||
|
||||
observation_types:
|
||||
- id: "bugfix"
|
||||
label: "Bug Fix"
|
||||
description: "Something was broken, now fixed"
|
||||
emoji: "🔴"
|
||||
work_emoji: "🛠️"
|
||||
- id: "feature"
|
||||
label: "Feature"
|
||||
description: "New capability or functionality added"
|
||||
emoji: "🟣"
|
||||
work_emoji: "🛠️"
|
||||
- id: "refactor"
|
||||
label: "Refactor"
|
||||
description: "Code restructured, behavior unchanged"
|
||||
emoji: "🔄"
|
||||
work_emoji: "🛠️"
|
||||
- id: "change"
|
||||
label: "Change"
|
||||
description: "Generic modification (docs, config, misc)"
|
||||
emoji: "✅"
|
||||
work_emoji: "🛠️"
|
||||
- id: "discovery"
|
||||
label: "Discovery"
|
||||
description: "Learning about existing system"
|
||||
emoji: "🔵"
|
||||
work_emoji: "🔍"
|
||||
- id: "decision"
|
||||
label: "Decision"
|
||||
description: "Architectural/design choice with rationale"
|
||||
emoji: "⚖️"
|
||||
work_emoji: "⚖️"
|
||||
|
||||
observation_concepts:
|
||||
- id: "how-it-works"
|
||||
label: "How It Works"
|
||||
description: "Understanding mechanisms"
|
||||
- id: "why-it-exists"
|
||||
label: "Why It Exists"
|
||||
description: "Purpose or rationale"
|
||||
- id: "what-changed"
|
||||
label: "What Changed"
|
||||
description: "Modifications made"
|
||||
- id: "problem-solution"
|
||||
label: "Problem-Solution"
|
||||
description: "Issues and their fixes"
|
||||
- id: "gotcha"
|
||||
label: "Gotcha"
|
||||
description: "Traps or edge cases"
|
||||
- id: "pattern"
|
||||
label: "Pattern"
|
||||
description: "Reusable approach"
|
||||
- id: "trade-off"
|
||||
label: "Trade-Off"
|
||||
description: "Pros/cons of a decision"
|
||||
|
||||
prompts:
|
||||
observer_role: |
|
||||
${observerRole}
|
||||
|
||||
recording_focus: |
|
||||
${recordingFocus}
|
||||
|
||||
skip_guidance: |
|
||||
${skipGuidance}
|
||||
|
||||
type_guidance: |
|
||||
${typeGuidance}
|
||||
|
||||
concept_guidance: |
|
||||
${conceptGuidance}
|
||||
|
||||
field_guidance: |
|
||||
${fieldGuidance}
|
||||
|
||||
format_examples: ""
|
||||
`;
|
||||
|
||||
// Write to modes/code.json
|
||||
const outputPath = path.join(__dirname, '../modes/code.json');
|
||||
fs.writeFileSync(outputPath, JSON.stringify(jsonData, null, 2), 'utf-8');
|
||||
|
||||
console.log('✅ Generated modes/code.json from prompts.ts');
|
||||
console.log('\nExtracted sections:');
|
||||
console.log('- observer_role:', observerRole.substring(0, 50) + '...');
|
||||
console.log('- recording_focus:', recordingFocus.substring(0, 50) + '...');
|
||||
console.log('- skip_guidance:', skipGuidance.substring(0, 50) + '...');
|
||||
console.log('- type_guidance:', typeGuidance.substring(0, 50) + '...');
|
||||
console.log('- concept_guidance:', conceptGuidance.substring(0, 50) + '...');
|
||||
console.log('- field_guidance:', fieldGuidance.substring(0, 50) + '...');
|
||||
@@ -1,177 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Extract Rich Context Examples
|
||||
* Shows what data we have available for memory worker using TranscriptParser API
|
||||
*/
|
||||
|
||||
import { TranscriptParser } from '../src/utils/transcript-parser.js';
|
||||
import { writeFileSync } from 'fs';
|
||||
import type { AssistantTranscriptEntry, UserTranscriptEntry } from '../src/types/transcript.js';
|
||||
|
||||
const transcriptPath = process.argv[2];
|
||||
|
||||
if (!transcriptPath) {
|
||||
console.error('Usage: tsx scripts/extract-rich-context-examples.ts <path-to-transcript.jsonl>');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const parser = new TranscriptParser(transcriptPath);
|
||||
|
||||
let output = '# Rich Context Examples\n\n';
|
||||
output += 'This document shows what contextual data is available in transcripts\n';
|
||||
output += 'that could improve observation generation quality.\n\n';
|
||||
|
||||
// Get stats using parser API
|
||||
const stats = parser.getParseStats();
|
||||
const tokens = parser.getTotalTokenUsage();
|
||||
|
||||
output += `## Statistics\n\n`;
|
||||
output += `- Total entries: ${stats.parsedEntries}\n`;
|
||||
output += `- User messages: ${stats.entriesByType['user'] || 0}\n`;
|
||||
output += `- Assistant messages: ${stats.entriesByType['assistant'] || 0}\n`;
|
||||
output += `- Token usage: ${(tokens.inputTokens + tokens.outputTokens).toLocaleString()} total\n`;
|
||||
output += `- Cache efficiency: ${tokens.cacheReadTokens.toLocaleString()} tokens read from cache\n\n`;
|
||||
|
||||
// Extract conversation pairs with tool uses
|
||||
const assistantEntries = parser.getAssistantEntries();
|
||||
const userEntries = parser.getUserEntries();
|
||||
|
||||
output += `## Conversation Flow\n\n`;
|
||||
output += `This shows how user requests, assistant reasoning, and tool executions flow together.\n`;
|
||||
output += `This is the rich context currently missing from individual tool observations.\n\n`;
|
||||
|
||||
let examplesFound = 0;
|
||||
const maxExamples = 5;
|
||||
|
||||
// Match assistant entries with their preceding user message
|
||||
for (let i = 0; i < assistantEntries.length && examplesFound < maxExamples; i++) {
|
||||
const assistantEntry = assistantEntries[i];
|
||||
const content = assistantEntry.message.content;
|
||||
|
||||
if (!Array.isArray(content)) continue;
|
||||
|
||||
// Extract components from assistant message
|
||||
const textBlocks = content.filter((c: any) => c.type === 'text');
|
||||
const thinkingBlocks = content.filter((c: any) => c.type === 'thinking');
|
||||
const toolUseBlocks = content.filter((c: any) => c.type === 'tool_use');
|
||||
|
||||
// Skip if no tools or only MCP tools
|
||||
const regularTools = toolUseBlocks.filter((t: any) =>
|
||||
!t.name.startsWith('mcp__')
|
||||
);
|
||||
|
||||
if (regularTools.length === 0) continue;
|
||||
|
||||
// Find the user message that preceded this assistant response
|
||||
let userMessage = '';
|
||||
const assistantTimestamp = new Date(assistantEntry.timestamp).getTime();
|
||||
|
||||
for (const userEntry of userEntries) {
|
||||
const userTimestamp = new Date(userEntry.timestamp).getTime();
|
||||
if (userTimestamp < assistantTimestamp) {
|
||||
// Extract user text using parser's helper
|
||||
const extractText = (content: any): string => {
|
||||
if (typeof content === 'string') return content;
|
||||
if (Array.isArray(content)) {
|
||||
return content
|
||||
.filter((c: any) => c.type === 'text')
|
||||
.map((c: any) => c.text)
|
||||
.join('\n');
|
||||
}
|
||||
return '';
|
||||
};
|
||||
|
||||
const text = extractText(userEntry.message.content);
|
||||
if (text.trim()) {
|
||||
userMessage = text;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
examplesFound++;
|
||||
output += `---\n\n`;
|
||||
output += `### Example ${examplesFound}\n\n`;
|
||||
|
||||
// 1. User Request
|
||||
if (userMessage) {
|
||||
output += `#### 👤 User Request\n`;
|
||||
const preview = userMessage.substring(0, 400);
|
||||
output += `\`\`\`\n${preview}${userMessage.length > 400 ? '\n...(truncated)' : ''}\n\`\`\`\n\n`;
|
||||
}
|
||||
|
||||
// 2. Assistant's Explanation (what it plans to do)
|
||||
if (textBlocks.length > 0) {
|
||||
const text = textBlocks.map((b: any) => b.text).join('\n');
|
||||
output += `#### 🤖 Assistant's Plan\n`;
|
||||
const preview = text.substring(0, 400);
|
||||
output += `\`\`\`\n${preview}${text.length > 400 ? '\n...(truncated)' : ''}\n\`\`\`\n\n`;
|
||||
}
|
||||
|
||||
// 3. Internal Reasoning (thinking)
|
||||
if (thinkingBlocks.length > 0) {
|
||||
const thinking = thinkingBlocks.map((b: any) => b.thinking).join('\n');
|
||||
output += `#### 💭 Internal Reasoning\n`;
|
||||
const preview = thinking.substring(0, 300);
|
||||
output += `\`\`\`\n${preview}${thinking.length > 300 ? '\n...(truncated)' : ''}\n\`\`\`\n\n`;
|
||||
}
|
||||
|
||||
// 4. Tool Executions
|
||||
output += `#### 🔧 Tools Executed (${regularTools.length})\n\n`;
|
||||
for (const tool of regularTools) {
|
||||
const toolData = tool as any;
|
||||
output += `**${toolData.name}**\n`;
|
||||
|
||||
// Show relevant input fields
|
||||
const input = toolData.input;
|
||||
if (toolData.name === 'Read') {
|
||||
output += `- Reading: \`${input.file_path}\`\n`;
|
||||
} else if (toolData.name === 'Write') {
|
||||
output += `- Writing: \`${input.file_path}\` (${input.content?.length || 0} chars)\n`;
|
||||
} else if (toolData.name === 'Edit') {
|
||||
output += `- Editing: \`${input.file_path}\`\n`;
|
||||
} else if (toolData.name === 'Bash') {
|
||||
output += `- Command: \`${input.command}\`\n`;
|
||||
} else if (toolData.name === 'Glob') {
|
||||
output += `- Pattern: \`${input.pattern}\`\n`;
|
||||
} else if (toolData.name === 'Grep') {
|
||||
output += `- Searching for: \`${input.pattern}\`\n`;
|
||||
} else {
|
||||
output += `\`\`\`json\n${JSON.stringify(input, null, 2).substring(0, 200)}\n\`\`\`\n`;
|
||||
}
|
||||
}
|
||||
output += `\n`;
|
||||
|
||||
// Summary of what data is available
|
||||
output += `**📊 Data Available for This Exchange:**\n`;
|
||||
output += `- User intent: ✅ (${userMessage.length} chars)\n`;
|
||||
output += `- Assistant reasoning: ✅ (${textBlocks.reduce((sum, b: any) => sum + b.text.length, 0)} chars)\n`;
|
||||
output += `- Thinking process: ${thinkingBlocks.length > 0 ? '✅' : '❌'} ${thinkingBlocks.length > 0 ? `(${thinkingBlocks.reduce((sum, b: any) => sum + b.thinking.length, 0)} chars)` : ''}\n`;
|
||||
output += `- Tool executions: ✅ (${regularTools.length} tools)\n`;
|
||||
output += `- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌\n\n`;
|
||||
}
|
||||
|
||||
output += `\n---\n\n`;
|
||||
output += `## Key Insight\n\n`;
|
||||
output += `Currently, the memory worker receives **isolated tool executions** via save-hook:\n`;
|
||||
output += `- tool_name: "Read"\n`;
|
||||
output += `- tool_input: {"file_path": "src/foo.ts"}\n`;
|
||||
output += `- tool_output: {file contents}\n\n`;
|
||||
output += `But the transcript contains **rich contextual data**:\n`;
|
||||
output += `- WHY the tool was used (user's request)\n`;
|
||||
output += `- WHAT the assistant planned to accomplish\n`;
|
||||
output += `- HOW it fits into the broader task\n`;
|
||||
output += `- The assistant's reasoning/thinking\n`;
|
||||
output += `- Multiple related tools used together\n\n`;
|
||||
output += `This context would help the memory worker:\n`;
|
||||
output += `1. Understand if a tool use is meaningful or routine\n`;
|
||||
output += `2. Generate observations that capture WHY, not just WHAT\n`;
|
||||
output += `3. Group related tools into coherent actions\n`;
|
||||
output += `4. Avoid "investigating" - the context is already present\n\n`;
|
||||
|
||||
// Write to file
|
||||
const outputPath = '/Users/alexnewman/Scripts/claude-mem/docs/context/rich-context-examples.md';
|
||||
writeFileSync(outputPath, output, 'utf-8');
|
||||
|
||||
console.log(`\nExtracted ${examplesFound} examples with rich context`);
|
||||
console.log(`Written to: ${outputPath}\n`);
|
||||
console.log(`This shows the gap between what's available (rich context) and what's sent (isolated tools)\n`);
|
||||
@@ -1,82 +0,0 @@
|
||||
# XML Extraction Scripts
|
||||
|
||||
Scripts to extract XML observations and summaries from Claude Code transcript files.
|
||||
|
||||
## Scripts
|
||||
|
||||
### `filter-actual-xml.py`
|
||||
**Recommended for import**
|
||||
|
||||
Extracts only actual XML from assistant responses, filtering out:
|
||||
- Template/example XML (with placeholders like `[...]` or `**field**:`)
|
||||
- XML from tool_use blocks
|
||||
- XML from user messages
|
||||
|
||||
**Output:** `~/Scripts/claude-mem/actual_xml_only_with_timestamps.xml`
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
python3 scripts/extraction/filter-actual-xml.py
|
||||
```
|
||||
|
||||
### `extract-all-xml.py`
|
||||
**For debugging/analysis**
|
||||
|
||||
Extracts ALL XML blocks from transcripts without filtering.
|
||||
|
||||
**Output:** `~/Scripts/claude-mem/all_xml_fragments_with_timestamps.xml`
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
python3 scripts/extraction/extract-all-xml.py
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Extract XML from transcripts:**
|
||||
```bash
|
||||
cd ~/Scripts/claude-mem
|
||||
python3 scripts/extraction/filter-actual-xml.py
|
||||
```
|
||||
|
||||
2. **Import to database:**
|
||||
```bash
|
||||
npm run import:xml
|
||||
```
|
||||
|
||||
3. **Clean up duplicates (if needed):**
|
||||
```bash
|
||||
npm run cleanup:duplicates
|
||||
```
|
||||
|
||||
## Source Data
|
||||
|
||||
Scripts read from: `~/.claude/projects/-Users-alexnewman-Scripts-claude-mem/*.jsonl`
|
||||
|
||||
These are Claude Code session transcripts stored in JSONL (JSON Lines) format.
|
||||
|
||||
## Output Format
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<transcript_extracts>
|
||||
|
||||
<!-- Block 1 | 2025-10-19 03:03:23 UTC -->
|
||||
<observation>
|
||||
<type>discovery</type>
|
||||
<title>Example observation</title>
|
||||
...
|
||||
</observation>
|
||||
|
||||
<!-- Block 2 | 2025-10-19 03:03:45 UTC -->
|
||||
<summary>
|
||||
<request>What was accomplished</request>
|
||||
...
|
||||
</summary>
|
||||
|
||||
</transcript_extracts>
|
||||
```
|
||||
|
||||
Each XML block includes a comment with:
|
||||
- Block number
|
||||
- Original timestamp from transcript
|
||||
@@ -1,128 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
import json
|
||||
import re
|
||||
from datetime import datetime
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
def extract_xml_blocks(text):
|
||||
"""Extract complete XML blocks from text"""
|
||||
xml_patterns = [
|
||||
r'<observation>.*?</observation>',
|
||||
r'<session_summary>.*?</session_summary>',
|
||||
r'<request>.*?</request>',
|
||||
r'<summary>.*?</summary>',
|
||||
r'<facts>.*?</facts>',
|
||||
r'<fact>.*?</fact>',
|
||||
r'<concepts>.*?</concepts>',
|
||||
r'<concept>.*?</concept>',
|
||||
r'<files>.*?</files>',
|
||||
r'<file>.*?</file>',
|
||||
r'<files_read>.*?</files_read>',
|
||||
r'<files_edited>.*?</files_edited>',
|
||||
r'<files_modified>.*?</files_modified>',
|
||||
r'<narrative>.*?</narrative>',
|
||||
r'<learned>.*?</learned>',
|
||||
r'<investigated>.*?</investigated>',
|
||||
r'<completed>.*?</completed>',
|
||||
r'<next_steps>.*?</next_steps>',
|
||||
r'<notes>.*?</notes>',
|
||||
r'<title>.*?</title>',
|
||||
r'<subtitle>.*?</subtitle>',
|
||||
r'<text>.*?</text>',
|
||||
r'<type>.*?</type>',
|
||||
]
|
||||
|
||||
blocks = []
|
||||
for pattern in xml_patterns:
|
||||
matches = re.findall(pattern, text, re.DOTALL)
|
||||
blocks.extend(matches)
|
||||
|
||||
return blocks
|
||||
|
||||
def process_transcript_file(filepath):
|
||||
"""Process a single transcript file and extract XML with timestamps"""
|
||||
results = []
|
||||
|
||||
with open(filepath, 'r', encoding='utf-8', errors='ignore') as f:
|
||||
for line in f:
|
||||
try:
|
||||
data = json.loads(line)
|
||||
|
||||
# Get timestamp
|
||||
timestamp = data.get('timestamp', 'unknown')
|
||||
|
||||
# Extract text content from message
|
||||
message = data.get('message', {})
|
||||
content = message.get('content', [])
|
||||
|
||||
if isinstance(content, list):
|
||||
for item in content:
|
||||
if isinstance(item, dict):
|
||||
text = ''
|
||||
if item.get('type') == 'text':
|
||||
text = item.get('text', '')
|
||||
elif item.get('type') == 'tool_use':
|
||||
# Also check tool_use input fields
|
||||
tool_input = item.get('input', {})
|
||||
if isinstance(tool_input, dict):
|
||||
text = str(tool_input)
|
||||
|
||||
if text:
|
||||
# Extract XML blocks
|
||||
xml_blocks = extract_xml_blocks(text)
|
||||
|
||||
for block in xml_blocks:
|
||||
results.append({
|
||||
'timestamp': timestamp,
|
||||
'xml': block
|
||||
})
|
||||
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
|
||||
return results
|
||||
|
||||
# Get list of transcript files
|
||||
transcript_dir = os.path.expanduser('~/.claude/projects/-Users-alexnewman-Scripts-claude-mem/')
|
||||
os.chdir(transcript_dir)
|
||||
|
||||
# Get all transcript files sorted by modification time
|
||||
result = subprocess.run(['ls', '-t'], capture_output=True, text=True)
|
||||
files = [f for f in result.stdout.strip().split('\n') if f.endswith('.jsonl')][:62]
|
||||
|
||||
all_results = []
|
||||
for filename in files:
|
||||
filepath = os.path.join(transcript_dir, filename)
|
||||
print(f"Processing {filename}...")
|
||||
results = process_transcript_file(filepath)
|
||||
all_results.extend(results)
|
||||
print(f" Found {len(results)} XML blocks")
|
||||
|
||||
# Write results with timestamps
|
||||
output_file = os.path.expanduser('~/Scripts/claude-mem/all_xml_fragments_with_timestamps.xml')
|
||||
with open(output_file, 'w', encoding='utf-8') as f:
|
||||
f.write('<?xml version="1.0" encoding="UTF-8"?>\n')
|
||||
f.write('<transcript_extracts>\n\n')
|
||||
|
||||
for i, item in enumerate(all_results, 1):
|
||||
timestamp = item['timestamp']
|
||||
xml = item['xml']
|
||||
|
||||
# Format timestamp nicely if it's ISO format
|
||||
if timestamp != 'unknown' and timestamp:
|
||||
try:
|
||||
dt = datetime.fromisoformat(timestamp.replace('Z', '+00:00'))
|
||||
formatted_time = dt.strftime('%Y-%m-%d %H:%M:%S UTC')
|
||||
except:
|
||||
formatted_time = timestamp
|
||||
else:
|
||||
formatted_time = 'unknown'
|
||||
|
||||
f.write(f'<!-- Block {i} | {formatted_time} -->\n')
|
||||
f.write(xml)
|
||||
f.write('\n\n')
|
||||
|
||||
f.write('</transcript_extracts>\n')
|
||||
|
||||
print(f"\nExtracted {len(all_results)} XML blocks with timestamps to {output_file}")
|
||||
@@ -1,168 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
import json
|
||||
import re
|
||||
from datetime import datetime
|
||||
import os
|
||||
|
||||
def extract_xml_blocks(text):
|
||||
"""Extract complete XML blocks from text"""
|
||||
xml_patterns = [
|
||||
r'<observation>.*?</observation>',
|
||||
r'<session_summary>.*?</session_summary>',
|
||||
r'<request>.*?</request>',
|
||||
r'<summary>.*?</summary>',
|
||||
r'<facts>.*?</facts>',
|
||||
r'<fact>.*?</fact>',
|
||||
r'<concepts>.*?</concepts>',
|
||||
r'<concept>.*?</concept>',
|
||||
r'<files>.*?</files>',
|
||||
r'<file>.*?</file>',
|
||||
r'<files_read>.*?</files_read>',
|
||||
r'<files_edited>.*?</files_edited>',
|
||||
r'<files_modified>.*?</files_modified>',
|
||||
r'<narrative>.*?</narrative>',
|
||||
r'<learned>.*?</learned>',
|
||||
r'<investigated>.*?</investigated>',
|
||||
r'<completed>.*?</completed>',
|
||||
r'<next_steps>.*?</next_steps>',
|
||||
r'<notes>.*?</notes>',
|
||||
r'<title>.*?</title>',
|
||||
r'<subtitle>.*?</subtitle>',
|
||||
r'<text>.*?</text>',
|
||||
r'<type>.*?</type>',
|
||||
r'<tool_used>.*?</tool_used>',
|
||||
r'<tool_name>.*?</tool_name>',
|
||||
r'<tool_input>.*?</tool_input>',
|
||||
r'<tool_output>.*?</tool_output>',
|
||||
r'<tool_time>.*?</tool_time>',
|
||||
]
|
||||
|
||||
blocks = []
|
||||
for pattern in xml_patterns:
|
||||
matches = re.findall(pattern, text, re.DOTALL)
|
||||
blocks.extend(matches)
|
||||
|
||||
return blocks
|
||||
|
||||
def is_example_xml(xml_block):
|
||||
"""Check if XML block is an example/template"""
|
||||
# Patterns that indicate this is example/template XML
|
||||
example_indicators = [
|
||||
r'\[.*?\]', # Square brackets with placeholders
|
||||
r'\*\*\w+\*\*:', # Bold markdown like **title**:
|
||||
r'\.\.\..*?\.\.\.', # Ellipsis indicating placeholder
|
||||
r'feature\|bugfix\|refactor', # Multiple options separated by |
|
||||
r'change \| discovery \| decision', # Example types
|
||||
r'\{.*?\}', # Curly braces (template variables)
|
||||
r'Concise, self-contained statement', # Literal example text
|
||||
r'Short title capturing',
|
||||
r'One sentence explanation',
|
||||
r'What was the user trying',
|
||||
r'What code/systems did you explore',
|
||||
r'What did you learn',
|
||||
r'What was done',
|
||||
r'What should happen next',
|
||||
r'file1\.ts', # Example filenames
|
||||
r'file2\.ts',
|
||||
r'file3\.ts',
|
||||
r'Any additional context',
|
||||
]
|
||||
|
||||
for pattern in example_indicators:
|
||||
if re.search(pattern, xml_block):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def process_transcript_file(filepath):
|
||||
"""Process a single transcript file and extract only real XML from assistant responses"""
|
||||
results = []
|
||||
|
||||
with open(filepath, 'r', encoding='utf-8', errors='ignore') as f:
|
||||
for line in f:
|
||||
try:
|
||||
data = json.loads(line)
|
||||
|
||||
# Get timestamp
|
||||
timestamp = data.get('timestamp', 'unknown')
|
||||
|
||||
# Only process assistant messages
|
||||
message = data.get('message', {})
|
||||
role = message.get('role')
|
||||
|
||||
if role != 'assistant':
|
||||
continue
|
||||
|
||||
content = message.get('content', [])
|
||||
|
||||
if isinstance(content, list):
|
||||
for item in content:
|
||||
if isinstance(item, dict) and item.get('type') == 'text':
|
||||
# This is text in an assistant response, not tool_use
|
||||
text = item.get('text', '')
|
||||
|
||||
# Extract XML blocks
|
||||
xml_blocks = extract_xml_blocks(text)
|
||||
|
||||
for block in xml_blocks:
|
||||
# Filter out example/template XML
|
||||
if not is_example_xml(block):
|
||||
results.append({
|
||||
'timestamp': timestamp,
|
||||
'xml': block
|
||||
})
|
||||
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
|
||||
return results
|
||||
|
||||
# Get list of Oct 18 transcript files
|
||||
import subprocess
|
||||
|
||||
transcript_dir = os.path.expanduser('~/.claude/projects/-Users-alexnewman-Scripts-claude-mem/')
|
||||
os.chdir(transcript_dir)
|
||||
|
||||
# Get all transcript files sorted by modification time
|
||||
result = subprocess.run(['ls', '-t'], capture_output=True, text=True)
|
||||
files = [f for f in result.stdout.strip().split('\n') if f.endswith('.jsonl')][:62]
|
||||
|
||||
all_results = []
|
||||
for filename in files:
|
||||
filepath = os.path.join(transcript_dir, filename)
|
||||
print(f"Processing {filename}...")
|
||||
results = process_transcript_file(filepath)
|
||||
all_results.extend(results)
|
||||
print(f" Found {len(results)} actual XML blocks")
|
||||
|
||||
# Write results with timestamps
|
||||
output_file = os.path.expanduser('~/Scripts/claude-mem/actual_xml_only_with_timestamps.xml')
|
||||
with open(output_file, 'w', encoding='utf-8') as f:
|
||||
f.write('<?xml version="1.0" encoding="UTF-8"?>\n')
|
||||
f.write('<!-- Actual XML blocks from assistant responses only -->\n')
|
||||
f.write('<!-- Excludes: tool_use inputs, user prompts, and example/template XML -->\n')
|
||||
f.write('<transcript_extracts>\n\n')
|
||||
|
||||
for i, item in enumerate(all_results, 1):
|
||||
timestamp = item['timestamp']
|
||||
xml = item['xml']
|
||||
|
||||
# Format timestamp nicely if it's ISO format
|
||||
if timestamp != 'unknown' and timestamp:
|
||||
try:
|
||||
dt = datetime.fromisoformat(timestamp.replace('Z', '+00:00'))
|
||||
formatted_time = dt.strftime('%Y-%m-%d %H:%M:%S UTC')
|
||||
except:
|
||||
formatted_time = timestamp
|
||||
else:
|
||||
formatted_time = 'unknown'
|
||||
|
||||
f.write(f'<!-- Block {i} | {formatted_time} -->\n')
|
||||
f.write(xml)
|
||||
f.write('\n\n')
|
||||
|
||||
f.write('</transcript_extracts>\n')
|
||||
|
||||
print(f"\n{'='*80}")
|
||||
print(f"Extracted {len(all_results)} actual XML blocks (filtered) to {output_file}")
|
||||
print(f"{'='*80}")
|
||||
@@ -1,38 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Find Silent Failure Patterns
|
||||
#
|
||||
# This script searches for defensive OR patterns (|| '' || null || undefined)
|
||||
# that should potentially use happy_path_error__with_fallback instead.
|
||||
#
|
||||
# Usage: ./scripts/find-silent-failures.sh
|
||||
|
||||
echo "=================================================="
|
||||
echo "Searching for defensive OR patterns in src/"
|
||||
echo "These MAY be silent failures that should log errors"
|
||||
echo "=================================================="
|
||||
echo ""
|
||||
|
||||
echo "🔍 Searching for: || ''"
|
||||
echo "---"
|
||||
grep -rn "|| ''" src/ --include="*.ts" --color=always || echo " (none found)"
|
||||
echo ""
|
||||
|
||||
echo "🔍 Searching for: || \"\""
|
||||
echo "---"
|
||||
grep -rn '|| ""' src/ --include="*.ts" --color=always || echo " (none found)"
|
||||
echo ""
|
||||
|
||||
echo "🔍 Searching for: || null"
|
||||
echo "---"
|
||||
grep -rn "|| null" src/ --include="*.ts" --color=always || echo " (none found)"
|
||||
echo ""
|
||||
|
||||
echo "🔍 Searching for: || undefined"
|
||||
echo "---"
|
||||
grep -rn "|| undefined" src/ --include="*.ts" --color=always || echo " (none found)"
|
||||
echo ""
|
||||
|
||||
echo "=================================================="
|
||||
echo "Review each match and determine if it should use:"
|
||||
echo " happy_path_error__with_fallback('description', data, fallback)"
|
||||
echo "=================================================="
|
||||
@@ -1,174 +0,0 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Fix ALL Corrupted Observation Timestamps
|
||||
*
|
||||
* This script finds and repairs ALL observations with timestamps that don't match
|
||||
* their session start times, not just ones in an arbitrary "bad window".
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
||||
|
||||
interface CorruptedObservation {
|
||||
obs_id: number;
|
||||
obs_title: string;
|
||||
obs_created: number;
|
||||
session_started: number;
|
||||
session_completed: number | null;
|
||||
memory_session_id: string;
|
||||
}
|
||||
|
||||
function formatTimestamp(epoch: number): string {
|
||||
return new Date(epoch).toLocaleString('en-US', {
|
||||
timeZone: 'America/Los_Angeles',
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
second: '2-digit'
|
||||
});
|
||||
}
|
||||
|
||||
function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const dryRun = args.includes('--dry-run');
|
||||
const autoYes = args.includes('--yes') || args.includes('-y');
|
||||
|
||||
console.log('🔍 Finding ALL observations with timestamp corruption...\n');
|
||||
if (dryRun) {
|
||||
console.log('🏃 DRY RUN MODE - No changes will be made\n');
|
||||
}
|
||||
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Find all observations where timestamp doesn't match session
|
||||
const corrupted = db.query<CorruptedObservation, []>(`
|
||||
SELECT
|
||||
o.id as obs_id,
|
||||
o.title as obs_title,
|
||||
o.created_at_epoch as obs_created,
|
||||
s.started_at_epoch as session_started,
|
||||
s.completed_at_epoch as session_completed,
|
||||
s.memory_session_id
|
||||
FROM observations o
|
||||
JOIN sdk_sessions s ON o.memory_session_id = s.memory_session_id
|
||||
WHERE o.created_at_epoch < s.started_at_epoch -- Observation older than session
|
||||
OR (s.completed_at_epoch IS NOT NULL
|
||||
AND o.created_at_epoch > (s.completed_at_epoch + 3600000)) -- More than 1hr after session
|
||||
ORDER BY o.id
|
||||
`).all();
|
||||
|
||||
console.log(`Found ${corrupted.length} observations with corrupted timestamps\n`);
|
||||
|
||||
if (corrupted.length === 0) {
|
||||
console.log('✅ No corrupted timestamps found!');
|
||||
db.close();
|
||||
return;
|
||||
}
|
||||
|
||||
// Display findings
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('PROPOSED FIXES:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════\n');
|
||||
|
||||
for (const obs of corrupted.slice(0, 50)) {
|
||||
const daysDiff = Math.round((obs.obs_created - obs.session_started) / (1000 * 60 * 60 * 24));
|
||||
console.log(`Observation #${obs.obs_id}: ${obs.obs_title || '(no title)'}`);
|
||||
console.log(` ❌ Wrong: ${formatTimestamp(obs.obs_created)}`);
|
||||
console.log(` ✅ Correct: ${formatTimestamp(obs.session_started)}`);
|
||||
console.log(` 📅 Off by ${daysDiff} days\n`);
|
||||
}
|
||||
|
||||
if (corrupted.length > 50) {
|
||||
console.log(`... and ${corrupted.length - 50} more\n`);
|
||||
}
|
||||
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log(`Ready to fix ${corrupted.length} observations.`);
|
||||
|
||||
if (dryRun) {
|
||||
console.log('\n🏃 DRY RUN COMPLETE - No changes made.');
|
||||
console.log('Run without --dry-run flag to apply fixes.\n');
|
||||
db.close();
|
||||
return;
|
||||
}
|
||||
|
||||
if (autoYes) {
|
||||
console.log('Auto-confirming with --yes flag...\n');
|
||||
applyFixes(db, corrupted);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('Apply these fixes? (y/n): ');
|
||||
|
||||
const stdin = Bun.stdin.stream();
|
||||
const reader = stdin.getReader();
|
||||
|
||||
reader.read().then(({ value }) => {
|
||||
const response = new TextDecoder().decode(value).trim().toLowerCase();
|
||||
|
||||
if (response === 'y' || response === 'yes') {
|
||||
applyFixes(db, corrupted);
|
||||
} else {
|
||||
console.log('\n❌ Fixes cancelled. No changes made.');
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Error:', error);
|
||||
db.close();
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
function applyFixes(db: Database, corrupted: CorruptedObservation[]) {
|
||||
console.log('\n🔧 Applying fixes...\n');
|
||||
|
||||
const updateStmt = db.prepare(`
|
||||
UPDATE observations
|
||||
SET created_at_epoch = ?,
|
||||
created_at = datetime(?/1000, 'unixepoch')
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
let successCount = 0;
|
||||
let errorCount = 0;
|
||||
|
||||
for (const obs of corrupted) {
|
||||
try {
|
||||
updateStmt.run(
|
||||
obs.session_started,
|
||||
obs.session_started,
|
||||
obs.obs_id
|
||||
);
|
||||
successCount++;
|
||||
if (successCount % 10 === 0 || successCount <= 10) {
|
||||
console.log(`✅ Fixed observation #${obs.obs_id}`);
|
||||
}
|
||||
} catch (error) {
|
||||
errorCount++;
|
||||
console.error(`❌ Failed to fix observation #${obs.obs_id}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('RESULTS:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log(`✅ Successfully fixed: ${successCount}`);
|
||||
console.log(`❌ Failed: ${errorCount}`);
|
||||
console.log(`📊 Total processed: ${corrupted.length}\n`);
|
||||
|
||||
if (successCount > 0) {
|
||||
console.log('🎉 ALL timestamp corruption has been repaired!\n');
|
||||
}
|
||||
|
||||
db.close();
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -1,22 +1,12 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Fix Corrupted Observation Timestamps
|
||||
*
|
||||
* This script repairs observations that were created during the orphan queue processing
|
||||
* on Dec 24, 2025 between 19:45-20:31. These observations got Dec 24 timestamps instead
|
||||
* of their original timestamps from Dec 17-20.
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
||||
|
||||
// Bad window: Dec 24 19:45-20:31 (timestamps in milliseconds, not microseconds)
|
||||
// Using actual observation epoch format (microseconds since epoch)
|
||||
const BAD_WINDOW_START = 1766623500000; // Dec 24 19:45 PST
|
||||
const BAD_WINDOW_END = 1766626260000; // Dec 24 20:31 PST
|
||||
const BAD_WINDOW_START = 1766623500000;
|
||||
const BAD_WINDOW_END = 1766626260000;
|
||||
|
||||
interface AffectedObservation {
|
||||
id: number;
|
||||
@@ -72,7 +62,6 @@ function main() {
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Step 1: Find affected observations
|
||||
console.log('Step 1: Finding observations created during bad window...');
|
||||
const affectedObs = db.query<AffectedObservation, []>(`
|
||||
SELECT id, memory_session_id, created_at_epoch, title
|
||||
@@ -89,7 +78,6 @@ function main() {
|
||||
return;
|
||||
}
|
||||
|
||||
// Step 2: Find processed pending_messages from bad window
|
||||
console.log('Step 2: Finding pending messages processed during bad window...');
|
||||
const processedMessages = db.query<ProcessedMessage, []>(`
|
||||
SELECT id, session_db_id, tool_name, created_at_epoch, completed_at_epoch
|
||||
@@ -102,7 +90,6 @@ function main() {
|
||||
|
||||
console.log(`Found ${processedMessages.length} processed messages\n`);
|
||||
|
||||
// Step 3: Match observations to their session start times (simpler approach)
|
||||
console.log('Step 3: Matching observations to session start times...');
|
||||
const fixes: TimestampFix[] = [];
|
||||
|
||||
@@ -136,13 +123,12 @@ function main() {
|
||||
wrong_timestamp: row.obs_created,
|
||||
correct_timestamp: row.session_started,
|
||||
session_db_id: 0, // Not needed for this approach
|
||||
pending_message_id: 0 // Not needed for this approach
|
||||
pending_message_id: 0
|
||||
});
|
||||
}
|
||||
|
||||
console.log(`Identified ${fixes.length} observations to fix\n`);
|
||||
|
||||
// Step 5: Display what will be fixed
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('PROPOSED FIXES:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════\n');
|
||||
@@ -155,7 +141,6 @@ function main() {
|
||||
console.log(` 📅 Off by ${daysDiff} days\n`);
|
||||
}
|
||||
|
||||
// Step 6: Ask for confirmation
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log(`Ready to fix ${fixes.length} observations.`);
|
||||
|
||||
|
||||
@@ -1,240 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Format Transcript Context
|
||||
*
|
||||
* Parses a Claude Code transcript and formats it to show rich contextual data
|
||||
* that could be used for improved observation generation.
|
||||
*/
|
||||
|
||||
import { TranscriptParser } from '../src/utils/transcript-parser.js';
|
||||
import { writeFileSync } from 'fs';
|
||||
import { basename } from 'path';
|
||||
|
||||
interface ConversationTurn {
|
||||
turnNumber: number;
|
||||
userMessage?: {
|
||||
content: string;
|
||||
timestamp: string;
|
||||
};
|
||||
assistantMessage?: {
|
||||
textContent: string;
|
||||
thinkingContent?: string;
|
||||
toolUses: Array<{
|
||||
name: string;
|
||||
input: any;
|
||||
timestamp: string;
|
||||
}>;
|
||||
timestamp: string;
|
||||
};
|
||||
toolResults?: Array<{
|
||||
toolName: string;
|
||||
result: any;
|
||||
timestamp: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
function extractConversationTurns(parser: TranscriptParser): ConversationTurn[] {
|
||||
const entries = parser.getAllEntries();
|
||||
const turns: ConversationTurn[] = [];
|
||||
let currentTurn: ConversationTurn | null = null;
|
||||
let turnNumber = 0;
|
||||
|
||||
for (const entry of entries) {
|
||||
// User messages start a new turn
|
||||
if (entry.type === 'user') {
|
||||
// If previous turn exists, push it
|
||||
if (currentTurn) {
|
||||
turns.push(currentTurn);
|
||||
}
|
||||
|
||||
// Start new turn
|
||||
turnNumber++;
|
||||
currentTurn = {
|
||||
turnNumber,
|
||||
toolResults: []
|
||||
};
|
||||
|
||||
// Extract user text (skip tool results)
|
||||
if (typeof entry.content === 'string') {
|
||||
currentTurn.userMessage = {
|
||||
content: entry.content,
|
||||
timestamp: entry.timestamp
|
||||
};
|
||||
} else if (Array.isArray(entry.content)) {
|
||||
const textContent = entry.content
|
||||
.filter((c: any) => c.type === 'text')
|
||||
.map((c: any) => c.text)
|
||||
.join('\n');
|
||||
|
||||
if (textContent.trim()) {
|
||||
currentTurn.userMessage = {
|
||||
content: textContent,
|
||||
timestamp: entry.timestamp
|
||||
};
|
||||
}
|
||||
|
||||
// Extract tool results
|
||||
const toolResults = entry.content.filter((c: any) => c.type === 'tool_result');
|
||||
for (const result of toolResults) {
|
||||
currentTurn.toolResults!.push({
|
||||
toolName: result.tool_use_id || 'unknown',
|
||||
result: result.content,
|
||||
timestamp: entry.timestamp
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Assistant messages
|
||||
if (entry.type === 'assistant' && currentTurn) {
|
||||
if (!Array.isArray(entry.content)) continue;
|
||||
|
||||
const textBlocks = entry.content.filter((c: any) => c.type === 'text');
|
||||
const thinkingBlocks = entry.content.filter((c: any) => c.type === 'thinking');
|
||||
const toolUseBlocks = entry.content.filter((c: any) => c.type === 'tool_use');
|
||||
|
||||
currentTurn.assistantMessage = {
|
||||
textContent: textBlocks.map((c: any) => c.text).join('\n'),
|
||||
thinkingContent: thinkingBlocks.map((c: any) => c.thinking).join('\n'),
|
||||
toolUses: toolUseBlocks.map((t: any) => ({
|
||||
name: t.name,
|
||||
input: t.input,
|
||||
timestamp: entry.timestamp
|
||||
})),
|
||||
timestamp: entry.timestamp
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// Push last turn
|
||||
if (currentTurn) {
|
||||
turns.push(currentTurn);
|
||||
}
|
||||
|
||||
return turns;
|
||||
}
|
||||
|
||||
function formatTurnToMarkdown(turn: ConversationTurn): string {
|
||||
let md = '';
|
||||
|
||||
md += `## Turn ${turn.turnNumber}\n\n`;
|
||||
|
||||
// User message
|
||||
if (turn.userMessage) {
|
||||
md += `### 👤 User Request\n`;
|
||||
md += `**Time:** ${new Date(turn.userMessage.timestamp).toLocaleString()}\n\n`;
|
||||
md += '```\n';
|
||||
md += turn.userMessage.content.substring(0, 500);
|
||||
if (turn.userMessage.content.length > 500) {
|
||||
md += '\n... (truncated)';
|
||||
}
|
||||
md += '\n```\n\n';
|
||||
}
|
||||
|
||||
// Assistant response
|
||||
if (turn.assistantMessage) {
|
||||
md += `### 🤖 Assistant Response\n`;
|
||||
md += `**Time:** ${new Date(turn.assistantMessage.timestamp).toLocaleString()}\n\n`;
|
||||
|
||||
// Text content
|
||||
if (turn.assistantMessage.textContent.trim()) {
|
||||
md += '**Response:**\n```\n';
|
||||
md += turn.assistantMessage.textContent.substring(0, 500);
|
||||
if (turn.assistantMessage.textContent.length > 500) {
|
||||
md += '\n... (truncated)';
|
||||
}
|
||||
md += '\n```\n\n';
|
||||
}
|
||||
|
||||
// Thinking
|
||||
if (turn.assistantMessage.thinkingContent?.trim()) {
|
||||
md += '**Thinking:**\n```\n';
|
||||
md += turn.assistantMessage.thinkingContent.substring(0, 300);
|
||||
if (turn.assistantMessage.thinkingContent.length > 300) {
|
||||
md += '\n... (truncated)';
|
||||
}
|
||||
md += '\n```\n\n';
|
||||
}
|
||||
|
||||
// Tool uses
|
||||
if (turn.assistantMessage.toolUses.length > 0) {
|
||||
md += `**Tools Used:** ${turn.assistantMessage.toolUses.length}\n\n`;
|
||||
for (const tool of turn.assistantMessage.toolUses) {
|
||||
md += `- **${tool.name}**\n`;
|
||||
md += ` \`\`\`json\n`;
|
||||
const inputStr = JSON.stringify(tool.input, null, 2);
|
||||
md += inputStr.substring(0, 200);
|
||||
if (inputStr.length > 200) {
|
||||
md += '\n ... (truncated)';
|
||||
}
|
||||
md += '\n ```\n';
|
||||
}
|
||||
md += '\n';
|
||||
}
|
||||
}
|
||||
|
||||
// Tool results summary
|
||||
if (turn.toolResults && turn.toolResults.length > 0) {
|
||||
md += `**Tool Results:** ${turn.toolResults.length} results received\n\n`;
|
||||
}
|
||||
|
||||
md += '---\n\n';
|
||||
return md;
|
||||
}
|
||||
|
||||
function formatTranscriptToMarkdown(transcriptPath: string): string {
|
||||
const parser = new TranscriptParser(transcriptPath);
|
||||
const turns = extractConversationTurns(parser);
|
||||
const stats = parser.getParseStats();
|
||||
const tokens = parser.getTotalTokenUsage();
|
||||
|
||||
let md = `# Transcript Context Analysis\n\n`;
|
||||
md += `**File:** ${basename(transcriptPath)}\n`;
|
||||
md += `**Parsed:** ${new Date().toLocaleString()}\n\n`;
|
||||
|
||||
md += `## Statistics\n\n`;
|
||||
md += `- Total entries: ${stats.totalLines}\n`;
|
||||
md += `- Successfully parsed: ${stats.parsedEntries}\n`;
|
||||
md += `- Failed lines: ${stats.failedLines}\n`;
|
||||
md += `- Conversation turns: ${turns.length}\n\n`;
|
||||
|
||||
md += `## Token Usage\n\n`;
|
||||
md += `- Input tokens: ${tokens.inputTokens.toLocaleString()}\n`;
|
||||
md += `- Output tokens: ${tokens.outputTokens.toLocaleString()}\n`;
|
||||
md += `- Cache creation: ${tokens.cacheCreationTokens.toLocaleString()}\n`;
|
||||
md += `- Cache read: ${tokens.cacheReadTokens.toLocaleString()}\n`;
|
||||
const totalTokens = tokens.inputTokens + tokens.outputTokens;
|
||||
md += `- Total: ${totalTokens.toLocaleString()}\n\n`;
|
||||
|
||||
md += `---\n\n`;
|
||||
md += `# Conversation Turns\n\n`;
|
||||
|
||||
// Format each turn
|
||||
for (const turn of turns.slice(0, 20)) { // Limit to first 20 turns for readability
|
||||
md += formatTurnToMarkdown(turn);
|
||||
}
|
||||
|
||||
if (turns.length > 20) {
|
||||
md += `\n_... ${turns.length - 20} more turns omitted for brevity_\n`;
|
||||
}
|
||||
|
||||
return md;
|
||||
}
|
||||
|
||||
// Main execution
|
||||
const transcriptPath = process.argv[2];
|
||||
|
||||
if (!transcriptPath) {
|
||||
console.error('Usage: tsx scripts/format-transcript-context.ts <path-to-transcript.jsonl>');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log(`Parsing transcript: ${transcriptPath}`);
|
||||
|
||||
const markdown = formatTranscriptToMarkdown(transcriptPath);
|
||||
const outputPath = transcriptPath.replace('.jsonl', '-formatted.md');
|
||||
|
||||
writeFileSync(outputPath, markdown, 'utf-8');
|
||||
|
||||
console.log(`\nFormatted transcript written to: ${outputPath}`);
|
||||
console.log(`\nOpen with: cat "${outputPath}"\n`);
|
||||
@@ -0,0 +1,143 @@
|
||||
#!/usr/bin/env node
|
||||
import { Jimp } from 'jimp';
|
||||
import { writeFileSync, readdirSync, existsSync } from 'fs';
|
||||
import { deflateRawSync } from 'zlib';
|
||||
import { join, dirname } from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const repoRoot = join(__dirname, '..');
|
||||
|
||||
const FRAMES_DIR = process.env.FRAMES_DIR || '/tmp/cmem-banner-frames';
|
||||
const OUT = join(repoRoot, 'src/npx-cli/banner-frames.ts');
|
||||
|
||||
const COLS = 128;
|
||||
const VIDEO_ROWS = Math.round(COLS * (9 / 16) / 2);
|
||||
const ROWS = VIDEO_ROWS;
|
||||
const TOP_PAD = 0;
|
||||
const BOTTOM_PAD = 0;
|
||||
|
||||
const RAMP = ' .·~+=*x%$@#';
|
||||
const BLACK_FLOOR = 50;
|
||||
const WHITE_CEIL = 160;
|
||||
const HALO_MIN = 70;
|
||||
const HALO_MAX = 175;
|
||||
|
||||
function rasterize(img, gridW, gridH) {
|
||||
const resized = img.clone().resize({ w: gridW, h: gridH });
|
||||
const data = resized.bitmap.data;
|
||||
const density = new Float32Array(gridW * gridH);
|
||||
for (let cy = 0; cy < gridH; cy++) {
|
||||
for (let cx = 0; cx < gridW; cx++) {
|
||||
const idx = (cy * gridW + cx) * 4;
|
||||
const r = data[idx], g = data[idx + 1], b = data[idx + 2];
|
||||
density[cy * gridW + cx] = 0.2126 * r + 0.7152 * g + 0.0722 * b;
|
||||
}
|
||||
}
|
||||
return density;
|
||||
}
|
||||
|
||||
function densityToChar(d) {
|
||||
if (d <= BLACK_FLOOR) return ' ';
|
||||
const range = WHITE_CEIL - BLACK_FLOOR;
|
||||
const norm = Math.min(1, (d - BLACK_FLOOR) / range);
|
||||
const t = Math.pow(norm, 1.3);
|
||||
const idx = Math.min(RAMP.length - 1, Math.max(1, Math.round(t * (RAMP.length - 1))));
|
||||
return RAMP[idx];
|
||||
}
|
||||
|
||||
function renderASCII(density, w, h) {
|
||||
const lines = [];
|
||||
for (let y = 0; y < h; y++) {
|
||||
let line = '';
|
||||
let inSpan = false;
|
||||
for (let x = 0; x < w; x++) {
|
||||
const i = y * w + x;
|
||||
const d = density[i];
|
||||
const ch = densityToChar(d);
|
||||
const wantSpan = d > HALO_MIN && d < HALO_MAX && ch !== ' ';
|
||||
if (wantSpan && !inSpan) { line += '<span>'; inSpan = true; }
|
||||
if (!wantSpan && inSpan) { line += '</span>'; inSpan = false; }
|
||||
line += ch;
|
||||
}
|
||||
if (inSpan) line += '</span>';
|
||||
lines.push(line);
|
||||
}
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
async function main() {
|
||||
if (!existsSync(FRAMES_DIR)) {
|
||||
throw new Error(`Frames directory not found: ${FRAMES_DIR}\n` +
|
||||
`Run: ffmpeg -y -i <video> -vf "scale=320:180" ${FRAMES_DIR}/frame_%04d.png`);
|
||||
}
|
||||
const files = readdirSync(FRAMES_DIR)
|
||||
.filter((f) => f.endsWith('.png'))
|
||||
.sort();
|
||||
if (files.length === 0) {
|
||||
throw new Error(`No PNG frames found in ${FRAMES_DIR}`);
|
||||
}
|
||||
|
||||
const blankLine = ' '.repeat(COLS);
|
||||
const topPadding = Array(TOP_PAD).fill(blankLine).join('\n');
|
||||
const bottomPadding = Array(BOTTOM_PAD).fill(blankLine).join('\n');
|
||||
|
||||
const frameStrings = [];
|
||||
for (let i = 0; i < files.length; i++) {
|
||||
const img = await Jimp.read(join(FRAMES_DIR, files[i]));
|
||||
const density = rasterize(img, COLS, VIDEO_ROWS);
|
||||
const body = renderASCII(density, COLS, VIDEO_ROWS);
|
||||
const padded = [topPadding, body, bottomPadding].filter(Boolean).join('\n');
|
||||
frameStrings.push(padded);
|
||||
if ((i + 1) % 32 === 0 || i === files.length - 1) {
|
||||
process.stdout.write(` rasterized ${i + 1}/${files.length}\r`);
|
||||
}
|
||||
}
|
||||
process.stdout.write('\n');
|
||||
|
||||
const joined = frameStrings.join('\x01');
|
||||
const compressed = deflateRawSync(Buffer.from(joined, 'utf8'), { level: 9 });
|
||||
const b64 = compressed.toString('base64');
|
||||
|
||||
const FRAME_DELAY = 22;
|
||||
|
||||
const ts = `// @strip-comments-keep — auto-generated, do not edit by hand.
|
||||
// Source: scripts/generate-banner-frames.mjs (webm video → ASCII via luminance ramp).
|
||||
// Frames are gzip-deflated, base64-encoded, separated by \\x01.
|
||||
|
||||
export interface BannerData {
|
||||
/** Base64-encoded raw deflate of all frames joined by \\x01 */
|
||||
compressed: string;
|
||||
frameCount: number;
|
||||
width: number;
|
||||
height: number;
|
||||
/** Milliseconds per frame */
|
||||
frameDelay: number;
|
||||
}
|
||||
|
||||
export const BANNER: BannerData = {
|
||||
compressed: ${JSON.stringify(b64)},
|
||||
frameCount: ${files.length},
|
||||
width: ${COLS},
|
||||
height: ${ROWS},
|
||||
frameDelay: ${FRAME_DELAY},
|
||||
};
|
||||
`;
|
||||
|
||||
writeFileSync(OUT, ts);
|
||||
console.log(`✓ Generated ${files.length} ASCII frames at ${COLS}×${ROWS}`);
|
||||
console.log(` Raw size: ${joined.length} bytes`);
|
||||
console.log(` Compressed: ${compressed.length} bytes (${((compressed.length / joined.length) * 100).toFixed(1)}%)`);
|
||||
console.log(` Base64: ${b64.length} bytes`);
|
||||
console.log(` Written to: ${OUT}`);
|
||||
|
||||
if (process.env.PREVIEW) {
|
||||
console.log('\n--- final frame preview ---');
|
||||
console.log(frameStrings[frameStrings.length - 1].replace(/<\/?span>/g, ''));
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error(err);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,14 +1,5 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Generate CHANGELOG.md from GitHub releases.
|
||||
*
|
||||
* Incremental by default: reads existing CHANGELOG.md, only fetches releases
|
||||
* newer than the newest version already documented, and prepends them.
|
||||
*
|
||||
* Pass --full to force a complete regeneration from every release.
|
||||
*/
|
||||
|
||||
import { execSync } from 'child_process';
|
||||
import { writeFileSync, readFileSync, existsSync } from 'fs';
|
||||
|
||||
@@ -69,11 +60,6 @@ function renderEntry(release) {
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the existing CHANGELOG.md and return:
|
||||
* - knownVersions: Set of version strings already present
|
||||
* - body: the content following the standard header (entries only)
|
||||
*/
|
||||
function readExistingChangelog() {
|
||||
if (!existsSync(CHANGELOG_PATH)) {
|
||||
return { knownVersions: new Set(), body: '' };
|
||||
@@ -85,7 +71,6 @@ function readExistingChangelog() {
|
||||
while ((match = versionHeaderRe.exec(content)) !== null) {
|
||||
knownVersions.add(match[1]);
|
||||
}
|
||||
// Strip the standard header so we can re-emit it cleanly
|
||||
const firstEntryIndex = content.search(/^## \[/m);
|
||||
const body = firstEntryIndex === -1 ? '' : content.slice(firstEntryIndex);
|
||||
return { knownVersions, body };
|
||||
|
||||
@@ -1,11 +1,4 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Import memories from a JSON export file with duplicate prevention
|
||||
* Usage: npx tsx scripts/import-memories.ts <input-file>
|
||||
* Example: npx tsx scripts/import-memories.ts windows-memories.json
|
||||
*
|
||||
* This script uses the worker API instead of direct database access.
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
|
||||
@@ -18,7 +11,6 @@ async function importMemories(inputFile: string) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Read and parse export file
|
||||
const exportData = JSON.parse(readFileSync(inputFile, 'utf-8'));
|
||||
|
||||
console.log(`📦 Import file: ${inputFile}`);
|
||||
@@ -31,7 +23,6 @@ async function importMemories(inputFile: string) {
|
||||
console.log(` • ${exportData.totalPrompts} prompts`);
|
||||
console.log('');
|
||||
|
||||
// Check if worker is running
|
||||
try {
|
||||
const healthCheck = await fetch(`${WORKER_URL}/api/stats`);
|
||||
if (!healthCheck.ok) {
|
||||
@@ -45,7 +36,6 @@ async function importMemories(inputFile: string) {
|
||||
|
||||
console.log('🔄 Importing via worker API...');
|
||||
|
||||
// Send import request to worker
|
||||
const response = await fetch(`${WORKER_URL}/api/import`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
@@ -77,7 +67,6 @@ async function importMemories(inputFile: string) {
|
||||
console.log(` Prompts: ${stats.promptsImported} imported, ${stats.promptsSkipped} skipped`);
|
||||
}
|
||||
|
||||
// CLI interface
|
||||
const args = process.argv.slice(2);
|
||||
if (args.length < 1) {
|
||||
console.error('Usage: npx tsx scripts/import-memories.ts <input-file>');
|
||||
|
||||
@@ -1,12 +1,5 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Investigate Timestamp Situation
|
||||
*
|
||||
* This script investigates the actual state of observations and pending messages
|
||||
* to understand what happened with the timestamp corruption.
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
@@ -30,10 +23,13 @@ function main() {
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Check 1: Recent observations on Dec 24
|
||||
console.log('Check 1: All observations created on Dec 24, 2025...');
|
||||
const dec24Start = 1735027200000; // Dec 24 00:00 PST
|
||||
const dec24End = 1735113600000; // Dec 25 00:00 PST
|
||||
// Computed at runtime to avoid drift; note that Date.UTC returns the
|
||||
// UTC midnight epoch — this script formats with America/Los_Angeles, so
|
||||
// the boundaries are on UTC days, not Pacific days. That's intentional
|
||||
// here: we want a stable epoch window the SQL can compare against.
|
||||
const dec24Start = Date.UTC(2025, 11, 24);
|
||||
const dec24End = Date.UTC(2025, 11, 25);
|
||||
|
||||
const dec24Obs = db.query(`
|
||||
SELECT id, memory_session_id, created_at_epoch, title
|
||||
@@ -53,10 +49,9 @@ function main() {
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Check 2: Observations from Dec 17-20
|
||||
console.log('Check 2: Observations from Dec 17-20, 2025...');
|
||||
const dec17Start = 1734422400000; // Dec 17 00:00 PST
|
||||
const dec21Start = 1734768000000; // Dec 21 00:00 PST
|
||||
const dec17Start = Date.UTC(2025, 11, 17);
|
||||
const dec21Start = Date.UTC(2025, 11, 21);
|
||||
|
||||
const oldObs = db.query(`
|
||||
SELECT id, memory_session_id, created_at_epoch, title
|
||||
@@ -76,7 +71,6 @@ function main() {
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Check 3: Pending messages status
|
||||
console.log('Check 3: Pending messages status...');
|
||||
const statusCounts = db.query(`
|
||||
SELECT status, COUNT(*) as count
|
||||
@@ -90,7 +84,6 @@ function main() {
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Check 4: Old pending messages from Dec 17-20
|
||||
console.log('Check 4: Pending messages from Dec 17-20...');
|
||||
const oldMessages = db.query(`
|
||||
SELECT id, session_db_id, tool_name, status, created_at_epoch, completed_at_epoch
|
||||
@@ -112,7 +105,6 @@ function main() {
|
||||
console.log(` ... and ${oldMessages.length - 20} more`);
|
||||
}
|
||||
|
||||
// Check 5: Recently completed pending messages
|
||||
console.log('Check 5: Recently completed pending messages...');
|
||||
const recentCompleted = db.query(`
|
||||
SELECT id, session_db_id, tool_name, status, created_at_epoch, completed_at_epoch
|
||||
|
||||
@@ -1,10 +1,5 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Release script for claude-mem
|
||||
* Handles version bumping, building, and creating marketplace releases
|
||||
*/
|
||||
|
||||
import { exec } from 'child_process';
|
||||
import { promisify } from 'util';
|
||||
import fs from 'fs';
|
||||
@@ -23,7 +18,6 @@ async function publish() {
|
||||
try {
|
||||
console.log('📦 Claude-mem Marketplace Release Tool\n');
|
||||
|
||||
// Check git status
|
||||
console.log('🔍 Checking git status...');
|
||||
const { stdout: gitStatus } = await execAsync('git status --porcelain');
|
||||
if (gitStatus.trim()) {
|
||||
@@ -39,12 +33,10 @@ async function publish() {
|
||||
console.log('✓ Working directory clean');
|
||||
}
|
||||
|
||||
// Get current version
|
||||
const packageJson = JSON.parse(fs.readFileSync('package.json', 'utf-8'));
|
||||
const currentVersion = packageJson.version;
|
||||
console.log(`\n📌 Current version: ${currentVersion}`);
|
||||
|
||||
// Ask for version bump type
|
||||
console.log('\nVersion bump type:');
|
||||
console.log(' 1. patch (x.x.X) - Bug fixes');
|
||||
console.log(' 2. minor (x.X.0) - New features');
|
||||
@@ -82,7 +74,6 @@ async function publish() {
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Update package.json and marketplace.json versions
|
||||
console.log('\n📝 Updating package.json and marketplace.json...');
|
||||
packageJson.version = newVersion;
|
||||
fs.writeFileSync('package.json', JSON.stringify(packageJson, null, 2) + '\n');
|
||||
@@ -92,12 +83,10 @@ async function publish() {
|
||||
fs.writeFileSync('.claude-plugin/marketplace.json', JSON.stringify(marketplaceJson, null, 2) + '\n');
|
||||
console.log('✓ Versions updated in both files');
|
||||
|
||||
// Run build
|
||||
console.log('\n🔨 Building hooks...');
|
||||
await execAsync('npm run build');
|
||||
console.log('✓ Build complete');
|
||||
|
||||
// Run tests if they exist
|
||||
if (packageJson.scripts?.test) {
|
||||
console.log('\n🧪 Running tests...');
|
||||
try {
|
||||
@@ -114,7 +103,6 @@ async function publish() {
|
||||
}
|
||||
}
|
||||
|
||||
// Git commit and tag
|
||||
console.log('\n📌 Creating git commit and tag...');
|
||||
await execAsync('git add package.json .claude-plugin/marketplace.json plugin/');
|
||||
await execAsync(`git commit -m "chore: Release v${newVersion}
|
||||
@@ -124,7 +112,6 @@ https://github.com/thedotmack/claude-mem"`);
|
||||
await execAsync(`git tag v${newVersion}`);
|
||||
console.log(`✓ Created commit and tag v${newVersion}`);
|
||||
|
||||
// Push to git
|
||||
console.log('\n⬆️ Pushing to git...');
|
||||
await execAsync('git push');
|
||||
await execAsync('git push --tags');
|
||||
|
||||
@@ -1,20 +1,4 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* Regenerate CLAUDE.md files for folders in the current project
|
||||
*
|
||||
* Usage:
|
||||
* bun scripts/regenerate-claude-md.ts [--dry-run] [--clean]
|
||||
*
|
||||
* Options:
|
||||
* --dry-run Show what would be done without writing files
|
||||
* --clean Remove auto-generated CLAUDE.md files instead of regenerating
|
||||
*
|
||||
* Behavior:
|
||||
* - Scopes to current working directory (not entire database history)
|
||||
* - Uses git ls-files to respect .gitignore (skips node_modules, .git, etc.)
|
||||
* - Only processes folders that exist within the current project
|
||||
* - Filters database to current project observations only
|
||||
*/
|
||||
|
||||
import { Database } from 'bun:sqlite';
|
||||
import path from 'path';
|
||||
@@ -43,12 +27,10 @@ interface ObservationRow {
|
||||
discovery_tokens: number | null;
|
||||
}
|
||||
|
||||
// Import shared utilities
|
||||
import { formatTime, groupByDate } from '../src/shared/timeline-formatting.js';
|
||||
import { isDirectChild } from '../src/shared/path-utils.js';
|
||||
import { replaceTaggedContent } from '../src/utils/claude-md-utils.js';
|
||||
|
||||
// Type icon map (matches ModeManager)
|
||||
const TYPE_ICONS: Record<string, string> = {
|
||||
'bugfix': '🔴',
|
||||
'feature': '🟣',
|
||||
@@ -72,31 +54,26 @@ function estimateTokens(obs: ObservationRow): number {
|
||||
return Math.ceil(size / 4);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get tracked folders using git ls-files
|
||||
* This respects .gitignore and only returns folders within the project
|
||||
*/
|
||||
function getTrackedFolders(workingDir: string): Set<string> {
|
||||
const folders = new Set<string>();
|
||||
// Always include the project root — the git-ls-files walker only adds
|
||||
// ancestors of files, and the fallback walker only adds child directories,
|
||||
// so neither path is guaranteed to add `workingDir` itself.
|
||||
folders.add(workingDir);
|
||||
|
||||
try {
|
||||
// Get all tracked files using git ls-files
|
||||
const output = execSync('git ls-files', {
|
||||
cwd: workingDir,
|
||||
encoding: 'utf-8',
|
||||
maxBuffer: 50 * 1024 * 1024 // 50MB buffer for large repos
|
||||
maxBuffer: 50 * 1024 * 1024
|
||||
});
|
||||
|
||||
const files = output.trim().split('\n').filter(f => f);
|
||||
|
||||
for (const file of files) {
|
||||
// Get the absolute path, then extract directory
|
||||
const absPath = path.join(workingDir, file);
|
||||
let dir = path.dirname(absPath);
|
||||
|
||||
// Add all parent directories up to and including the working dir itself.
|
||||
// The working dir is included so that root-level files (stored in the DB
|
||||
// as bare filenames with no directory component) can be matched. Fixes #1514.
|
||||
while (dir.length >= workingDir.length && dir.startsWith(workingDir)) {
|
||||
folders.add(dir);
|
||||
if (dir === workingDir) break;
|
||||
@@ -105,18 +82,15 @@ function getTrackedFolders(workingDir: string): Set<string> {
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Warning: git ls-files failed, falling back to directory walk');
|
||||
// Fallback: walk directories but skip common ignored patterns
|
||||
walkDirectoriesWithIgnore(workingDir, folders);
|
||||
}
|
||||
|
||||
return folders;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback directory walker that skips common ignored patterns
|
||||
*/
|
||||
function walkDirectoriesWithIgnore(dir: string, folders: Set<string>, depth: number = 0): void {
|
||||
if (depth > 10) return; // Prevent infinite recursion
|
||||
if (depth > 10) return;
|
||||
folders.add(dir);
|
||||
|
||||
const ignorePatterns = [
|
||||
'node_modules', '.git', '.next', 'dist', 'build', '.cache',
|
||||
@@ -140,9 +114,6 @@ function walkDirectoriesWithIgnore(dir: string, folders: Set<string>, depth: num
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if an observation has any files that are direct children of the folder
|
||||
*/
|
||||
function hasDirectChildFile(obs: ObservationRow, folderPath: string): boolean {
|
||||
const checkFiles = (filesJson: string | null): boolean => {
|
||||
if (!filesJson) return false;
|
||||
@@ -158,20 +129,9 @@ function hasDirectChildFile(obs: ObservationRow, folderPath: string): boolean {
|
||||
return checkFiles(obs.files_modified) || checkFiles(obs.files_read);
|
||||
}
|
||||
|
||||
/**
|
||||
* Query observations for a specific folder
|
||||
* folderPath is a relative path from the project root (e.g., "src/services")
|
||||
* Only returns observations with files directly in the folder (not in subfolders)
|
||||
*/
|
||||
function findObservationsByFolder(db: Database, relativeFolderPath: string, project: string, limit: number): ObservationRow[] {
|
||||
// Query more results than needed since we'll filter some out
|
||||
const queryLimit = limit * 3;
|
||||
|
||||
// For the root folder (empty relativeFolderPath), observations may have bare
|
||||
// filenames stored without any directory component (e.g. ["dashboard.html"]).
|
||||
// In that case the LIKE pattern below would never match, so we fetch all
|
||||
// observations for the project and let isDirectChild filter to root-level files.
|
||||
// Fixes #1514.
|
||||
let allMatches: ObservationRow[];
|
||||
|
||||
if (relativeFolderPath === '' || relativeFolderPath === '.') {
|
||||
@@ -193,31 +153,20 @@ function findObservationsByFolder(db: Database, relativeFolderPath: string, proj
|
||||
ORDER BY o.created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`;
|
||||
// Files in DB are stored as relative paths like "src/services/foo.ts"
|
||||
// Match any file that starts with this folder path (we'll filter to direct children below)
|
||||
const likePattern = `%"${relativeFolderPath}/%`;
|
||||
allMatches = db.prepare(sql).all(project, likePattern, likePattern, queryLimit) as ObservationRow[];
|
||||
}
|
||||
|
||||
// Filter to only observations with direct child files (not in subfolders)
|
||||
return allMatches.filter(obs => hasDirectChildFile(obs, relativeFolderPath)).slice(0, limit);
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract relevant file from an observation for display
|
||||
* Only returns files that are direct children of the folder (not in subfolders)
|
||||
* @param obs - The observation row
|
||||
* @param relativeFolder - Relative folder path (e.g., "src/services")
|
||||
*/
|
||||
function extractRelevantFile(obs: ObservationRow, relativeFolder: string): string {
|
||||
// Try files_modified first - only direct children
|
||||
if (obs.files_modified) {
|
||||
try {
|
||||
const modified = JSON.parse(obs.files_modified);
|
||||
if (Array.isArray(modified) && modified.length > 0) {
|
||||
for (const file of modified) {
|
||||
if (isDirectChild(file, relativeFolder)) {
|
||||
// Get just the filename (no path since it's a direct child)
|
||||
return path.basename(file);
|
||||
}
|
||||
}
|
||||
@@ -225,7 +174,6 @@ function extractRelevantFile(obs: ObservationRow, relativeFolder: string): strin
|
||||
} catch {}
|
||||
}
|
||||
|
||||
// Fall back to files_read - only direct children
|
||||
if (obs.files_read) {
|
||||
try {
|
||||
const read = JSON.parse(obs.files_read);
|
||||
@@ -242,9 +190,6 @@ function extractRelevantFile(obs: ObservationRow, relativeFolder: string): strin
|
||||
return 'General';
|
||||
}
|
||||
|
||||
/**
|
||||
* Format observations for CLAUDE.md content
|
||||
*/
|
||||
function formatObservationsForClaudeMd(observations: ObservationRow[], folderPath: string): string {
|
||||
const lines: string[] = [];
|
||||
lines.push('# Recent Activity');
|
||||
@@ -292,54 +237,33 @@ function formatObservationsForClaudeMd(observations: ObservationRow[], folderPat
|
||||
return lines.join('\n').trim();
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Write CLAUDE.md file with tagged content preservation
|
||||
* Note: For the CLI regenerate tool, we DO create directories since the user
|
||||
* explicitly requested regeneration. This differs from the runtime behavior
|
||||
* which only writes to existing folders.
|
||||
*/
|
||||
function writeClaudeMdToFolderForRegenerate(folderPath: string, newContent: string): void {
|
||||
const resolvedPath = path.resolve(folderPath);
|
||||
|
||||
// Never write inside .git directories — corrupts refs (#1165)
|
||||
if (resolvedPath.includes('/.git/') || resolvedPath.includes('\\.git\\') || resolvedPath.endsWith('/.git') || resolvedPath.endsWith('\\.git')) return;
|
||||
|
||||
const claudeMdPath = path.join(folderPath, 'CLAUDE.md');
|
||||
const tempFile = `${claudeMdPath}.tmp`;
|
||||
|
||||
// For regenerate CLI, we create the folder if needed
|
||||
mkdirSync(folderPath, { recursive: true });
|
||||
|
||||
// Read existing content if file exists
|
||||
let existingContent = '';
|
||||
if (existsSync(claudeMdPath)) {
|
||||
existingContent = readFileSync(claudeMdPath, 'utf-8');
|
||||
}
|
||||
|
||||
// Use shared utility to preserve user content outside tags
|
||||
const finalContent = replaceTaggedContent(existingContent, newContent);
|
||||
|
||||
// Atomic write: temp file + rename
|
||||
writeFileSync(tempFile, finalContent);
|
||||
renameSync(tempFile, claudeMdPath);
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up auto-generated CLAUDE.md files
|
||||
*
|
||||
* For each file with <claude-mem-context> tags:
|
||||
* - Strip the tagged section
|
||||
* - If empty after stripping → delete the file
|
||||
* - If has remaining content → save the stripped version
|
||||
*/
|
||||
function cleanupAutoGeneratedFiles(workingDir: string, dryRun: boolean): void {
|
||||
console.log('=== CLAUDE.md Cleanup Mode ===\n');
|
||||
console.log(`Scanning ${workingDir} for CLAUDE.md files with auto-generated content...\n`);
|
||||
|
||||
const filesToProcess: string[] = [];
|
||||
|
||||
// Walk directories to find CLAUDE.md files
|
||||
function walkForClaudeMd(dir: string): void {
|
||||
const ignorePatterns = ['node_modules', '.git', '.next', 'dist', 'build'];
|
||||
|
||||
@@ -353,7 +277,6 @@ function cleanupAutoGeneratedFiles(workingDir: string, dryRun: boolean): void {
|
||||
walkForClaudeMd(fullPath);
|
||||
}
|
||||
} else if (entry.name === 'CLAUDE.md') {
|
||||
// Check if file contains auto-generated content
|
||||
try {
|
||||
const content = readFileSync(fullPath, 'utf-8');
|
||||
if (content.includes('<claude-mem-context>')) {
|
||||
@@ -388,11 +311,9 @@ function cleanupAutoGeneratedFiles(workingDir: string, dryRun: boolean): void {
|
||||
try {
|
||||
const content = readFileSync(file, 'utf-8');
|
||||
|
||||
// Strip the claude-mem-context tagged section
|
||||
const stripped = content.replace(/<claude-mem-context>[\s\S]*?<\/claude-mem-context>/g, '').trim();
|
||||
|
||||
if (stripped === '') {
|
||||
// Empty after stripping → delete
|
||||
if (dryRun) {
|
||||
console.log(` [DRY-RUN] Would delete (empty): ${relativePath}`);
|
||||
} else {
|
||||
@@ -401,7 +322,6 @@ function cleanupAutoGeneratedFiles(workingDir: string, dryRun: boolean): void {
|
||||
}
|
||||
deletedCount++;
|
||||
} else {
|
||||
// Has content → write stripped version
|
||||
if (dryRun) {
|
||||
console.log(` [DRY-RUN] Would clean: ${relativePath}`);
|
||||
} else {
|
||||
@@ -426,11 +346,6 @@ function cleanupAutoGeneratedFiles(workingDir: string, dryRun: boolean): void {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Regenerate CLAUDE.md for a single folder
|
||||
* @param absoluteFolder - Absolute path for writing files
|
||||
* @param relativeFolder - Relative path for DB queries (matches storage format)
|
||||
*/
|
||||
function regenerateFolder(
|
||||
db: Database,
|
||||
absoluteFolder: string,
|
||||
@@ -439,7 +354,6 @@ function regenerateFolder(
|
||||
dryRun: boolean
|
||||
): { success: boolean; observationCount: number; error?: string } {
|
||||
try {
|
||||
// Query using relative path (matches DB storage format)
|
||||
const observations = findObservationsByFolder(db, relativeFolder, project, OBSERVATION_LIMIT);
|
||||
|
||||
if (observations.length === 0) {
|
||||
@@ -450,7 +364,6 @@ function regenerateFolder(
|
||||
return { success: true, observationCount: observations.length };
|
||||
}
|
||||
|
||||
// Format using relative path for display, write to absolute path
|
||||
const formatted = formatObservationsForClaudeMd(observations, relativeFolder);
|
||||
writeClaudeMdToFolderForRegenerate(absoluteFolder, formatted);
|
||||
|
||||
@@ -460,9 +373,6 @@ function regenerateFolder(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Main function
|
||||
*/
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const dryRun = args.includes('--dry-run');
|
||||
@@ -470,7 +380,6 @@ async function main() {
|
||||
|
||||
const workingDir = process.cwd();
|
||||
|
||||
// Handle cleanup mode
|
||||
if (cleanMode) {
|
||||
cleanupAutoGeneratedFiles(workingDir, dryRun);
|
||||
return;
|
||||
@@ -479,11 +388,9 @@ async function main() {
|
||||
console.log('=== CLAUDE.md Regeneration Script ===\n');
|
||||
console.log(`Working directory: ${workingDir}`);
|
||||
|
||||
// Determine project identifier (matches how hooks determine project - uses folder name)
|
||||
const project = path.basename(workingDir);
|
||||
console.log(`Project: ${project}\n`);
|
||||
|
||||
// Get tracked folders using git ls-files
|
||||
console.log('Discovering folders (using git ls-files to respect .gitignore)...');
|
||||
const trackedFolders = getTrackedFolders(workingDir);
|
||||
|
||||
@@ -494,7 +401,6 @@ async function main() {
|
||||
|
||||
console.log(`Found ${trackedFolders.size} folders in project.\n`);
|
||||
|
||||
// Open database
|
||||
if (!existsSync(DB_PATH)) {
|
||||
console.log('Database not found. No observations to process.');
|
||||
process.exit(0);
|
||||
@@ -507,7 +413,6 @@ async function main() {
|
||||
console.log('[DRY RUN] Would regenerate the following folders:\n');
|
||||
}
|
||||
|
||||
// Process each folder
|
||||
let successCount = 0;
|
||||
let skipCount = 0;
|
||||
let errorCount = 0;
|
||||
@@ -520,7 +425,6 @@ async function main() {
|
||||
const relativeFolder = path.relative(workingDir, absoluteFolder);
|
||||
|
||||
if (dryRun) {
|
||||
// Query using relative path (matches DB storage format)
|
||||
const observations = findObservationsByFolder(db, relativeFolder, project, OBSERVATION_LIMIT);
|
||||
if (observations.length > 0) {
|
||||
console.log(`${progress} ${relativeFolder} (${observations.length} obs)`);
|
||||
@@ -546,7 +450,6 @@ async function main() {
|
||||
|
||||
db.close();
|
||||
|
||||
// Summary
|
||||
console.log('\n=== Summary ===');
|
||||
console.log(`Total folders scanned: ${foldersArray.length}`);
|
||||
console.log(`With observations: ${successCount}`);
|
||||
|
||||
@@ -1,325 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Smart Install Script for claude-mem
|
||||
*
|
||||
* Ensures Bun runtime and uv (Python package manager) are installed
|
||||
* (auto-installs if missing) and handles dependency installation when needed.
|
||||
*/
|
||||
import { existsSync, readFileSync, writeFileSync } from 'fs';
|
||||
import { execSync, spawnSync } from 'child_process';
|
||||
import { join, dirname } from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const IS_WINDOWS = process.platform === 'win32';
|
||||
|
||||
/**
|
||||
* Resolve the plugin root directory where dependencies should be installed.
|
||||
*
|
||||
* Priority:
|
||||
* 1. CLAUDE_PLUGIN_ROOT env var (set by Claude Code for hooks — works for
|
||||
* both cache-based and marketplace installs)
|
||||
* 2. Script location (dirname of this file, up one level from scripts/)
|
||||
* 3. XDG path (~/.config/claude/plugins/marketplaces/thedotmack)
|
||||
* 4. Legacy path (~/.claude/plugins/marketplaces/thedotmack)
|
||||
*/
|
||||
function resolveRoot() {
|
||||
// CLAUDE_PLUGIN_ROOT is the authoritative location set by Claude Code
|
||||
if (process.env.CLAUDE_PLUGIN_ROOT) {
|
||||
const root = process.env.CLAUDE_PLUGIN_ROOT;
|
||||
if (existsSync(join(root, 'package.json'))) return root;
|
||||
}
|
||||
|
||||
// Derive from script location (this file is in <root>/scripts/)
|
||||
try {
|
||||
const scriptDir = dirname(fileURLToPath(import.meta.url));
|
||||
const candidate = dirname(scriptDir);
|
||||
if (existsSync(join(candidate, 'package.json'))) return candidate;
|
||||
} catch {
|
||||
// import.meta.url not available
|
||||
}
|
||||
|
||||
// Probe XDG path, then legacy
|
||||
const marketplaceRel = join('plugins', 'marketplaces', 'thedotmack');
|
||||
const xdg = join(homedir(), '.config', 'claude', marketplaceRel);
|
||||
if (existsSync(join(xdg, 'package.json'))) return xdg;
|
||||
|
||||
return join(homedir(), '.claude', marketplaceRel);
|
||||
}
|
||||
|
||||
const ROOT = resolveRoot();
|
||||
const MARKER = join(ROOT, '.install-version');
|
||||
|
||||
// Common installation paths (handles fresh installs before PATH reload)
|
||||
const BUN_COMMON_PATHS = IS_WINDOWS
|
||||
? [join(homedir(), '.bun', 'bin', 'bun.exe')]
|
||||
: [join(homedir(), '.bun', 'bin', 'bun'), '/usr/local/bin/bun', '/opt/homebrew/bin/bun'];
|
||||
|
||||
const UV_COMMON_PATHS = IS_WINDOWS
|
||||
? [join(homedir(), '.local', 'bin', 'uv.exe'), join(homedir(), '.cargo', 'bin', 'uv.exe')]
|
||||
: [join(homedir(), '.local', 'bin', 'uv'), join(homedir(), '.cargo', 'bin', 'uv'), '/usr/local/bin/uv', '/opt/homebrew/bin/uv'];
|
||||
|
||||
/**
|
||||
* Get the Bun executable path (from PATH or common install locations)
|
||||
*/
|
||||
function getBunPath() {
|
||||
// Try PATH first
|
||||
try {
|
||||
const result = spawnSync('bun', ['--version'], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
shell: IS_WINDOWS
|
||||
});
|
||||
if (result.status === 0) return 'bun';
|
||||
} catch {
|
||||
// Not in PATH
|
||||
}
|
||||
|
||||
// Check common installation paths
|
||||
return BUN_COMMON_PATHS.find(existsSync) || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Bun is installed and accessible
|
||||
*/
|
||||
function isBunInstalled() {
|
||||
return getBunPath() !== null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get Bun version if installed
|
||||
*/
|
||||
function getBunVersion() {
|
||||
const bunPath = getBunPath();
|
||||
if (!bunPath) return null;
|
||||
|
||||
try {
|
||||
const result = spawnSync(bunPath, ['--version'], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
shell: IS_WINDOWS
|
||||
});
|
||||
return result.status === 0 ? result.stdout.trim() : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the uv executable path (from PATH or common install locations)
|
||||
*/
|
||||
function getUvPath() {
|
||||
// Try PATH first
|
||||
try {
|
||||
const result = spawnSync('uv', ['--version'], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
shell: IS_WINDOWS
|
||||
});
|
||||
if (result.status === 0) return 'uv';
|
||||
} catch {
|
||||
// Not in PATH
|
||||
}
|
||||
|
||||
// Check common installation paths
|
||||
return UV_COMMON_PATHS.find(existsSync) || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if uv is installed and accessible
|
||||
*/
|
||||
function isUvInstalled() {
|
||||
return getUvPath() !== null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get uv version if installed
|
||||
*/
|
||||
function getUvVersion() {
|
||||
const uvPath = getUvPath();
|
||||
if (!uvPath) return null;
|
||||
|
||||
try {
|
||||
const result = spawnSync(uvPath, ['--version'], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
shell: IS_WINDOWS
|
||||
});
|
||||
return result.status === 0 ? result.stdout.trim() : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Install Bun automatically based on platform
|
||||
*/
|
||||
function installBun() {
|
||||
console.error('🔧 Bun not found. Installing Bun runtime...');
|
||||
|
||||
try {
|
||||
if (IS_WINDOWS) {
|
||||
console.error(' Installing via PowerShell...');
|
||||
execSync('powershell -c "irm bun.sh/install.ps1 | iex"', {
|
||||
stdio: 'inherit',
|
||||
shell: true
|
||||
});
|
||||
} else {
|
||||
console.error(' Installing via curl...');
|
||||
execSync('curl -fsSL https://bun.sh/install | bash', {
|
||||
stdio: 'inherit',
|
||||
shell: true
|
||||
});
|
||||
}
|
||||
|
||||
if (!isBunInstalled()) {
|
||||
throw new Error(
|
||||
'Bun installation completed but binary not found. ' +
|
||||
'Please restart your terminal and try again.'
|
||||
);
|
||||
}
|
||||
|
||||
const version = getBunVersion();
|
||||
console.error(`✅ Bun ${version} installed successfully`);
|
||||
} catch (error) {
|
||||
console.error('❌ Failed to install Bun');
|
||||
console.error(' Please install manually:');
|
||||
if (IS_WINDOWS) {
|
||||
console.error(' - winget install Oven-sh.Bun');
|
||||
console.error(' - Or: powershell -c "irm bun.sh/install.ps1 | iex"');
|
||||
} else {
|
||||
console.error(' - curl -fsSL https://bun.sh/install | bash');
|
||||
console.error(' - Or: brew install oven-sh/bun/bun');
|
||||
}
|
||||
console.error(' Then restart your terminal and try again.');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Install uv automatically based on platform
|
||||
*/
|
||||
function installUv() {
|
||||
console.error('🐍 Installing uv for Python/Chroma support...');
|
||||
|
||||
try {
|
||||
if (IS_WINDOWS) {
|
||||
console.error(' Installing via PowerShell...');
|
||||
execSync('powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"', {
|
||||
stdio: 'inherit',
|
||||
shell: true
|
||||
});
|
||||
} else {
|
||||
console.error(' Installing via curl...');
|
||||
execSync('curl -LsSf https://astral.sh/uv/install.sh | sh', {
|
||||
stdio: 'inherit',
|
||||
shell: true
|
||||
});
|
||||
}
|
||||
|
||||
if (!isUvInstalled()) {
|
||||
throw new Error(
|
||||
'uv installation completed but binary not found. ' +
|
||||
'Please restart your terminal and try again.'
|
||||
);
|
||||
}
|
||||
|
||||
const version = getUvVersion();
|
||||
console.error(`✅ uv ${version} installed successfully`);
|
||||
} catch (error) {
|
||||
console.error('❌ Failed to install uv');
|
||||
console.error(' Please install manually:');
|
||||
if (IS_WINDOWS) {
|
||||
console.error(' - winget install astral-sh.uv');
|
||||
console.error(' - Or: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"');
|
||||
} else {
|
||||
console.error(' - curl -LsSf https://astral.sh/uv/install.sh | sh');
|
||||
console.error(' - Or: brew install uv (macOS)');
|
||||
}
|
||||
console.error(' Then restart your terminal and try again.');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if dependencies need to be installed
|
||||
*/
|
||||
function needsInstall() {
|
||||
if (!existsSync(join(ROOT, 'node_modules'))) return true;
|
||||
try {
|
||||
const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8'));
|
||||
const marker = JSON.parse(readFileSync(MARKER, 'utf-8'));
|
||||
return pkg.version !== marker.version || getBunVersion() !== marker.bun;
|
||||
} catch {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Install dependencies using Bun
|
||||
*/
|
||||
function installDeps() {
|
||||
const bunPath = getBunPath();
|
||||
if (!bunPath) {
|
||||
throw new Error('Bun executable not found');
|
||||
}
|
||||
|
||||
console.error('📦 Installing dependencies with Bun...');
|
||||
|
||||
// Quote path for Windows paths with spaces
|
||||
const bunCmd = IS_WINDOWS && bunPath.includes(' ') ? `"${bunPath}"` : bunPath;
|
||||
|
||||
execSync(`${bunCmd} install`, { cwd: ROOT, stdio: 'inherit', shell: IS_WINDOWS });
|
||||
|
||||
// Write version marker
|
||||
const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8'));
|
||||
writeFileSync(MARKER, JSON.stringify({
|
||||
version: pkg.version,
|
||||
bun: getBunVersion(),
|
||||
uv: getUvVersion(),
|
||||
installedAt: new Date().toISOString()
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify that critical runtime modules are resolvable from the install directory.
|
||||
* Returns true if all critical modules exist, false otherwise.
|
||||
*/
|
||||
function verifyCriticalModules() {
|
||||
const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8'));
|
||||
const dependencies = Object.keys(pkg.dependencies || {});
|
||||
|
||||
const missing = [];
|
||||
for (const dep of dependencies) {
|
||||
const modulePath = join(ROOT, 'node_modules', ...dep.split('/'));
|
||||
if (!existsSync(modulePath)) {
|
||||
missing.push(dep);
|
||||
}
|
||||
}
|
||||
|
||||
if (missing.length > 0) {
|
||||
console.error(`❌ Post-install check failed: missing modules: ${missing.join(', ')}`);
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
// Main execution
|
||||
try {
|
||||
if (!isBunInstalled()) installBun();
|
||||
if (!isUvInstalled()) installUv();
|
||||
if (needsInstall()) {
|
||||
installDeps();
|
||||
|
||||
if (!verifyCriticalModules()) {
|
||||
console.error('❌ Dependencies could not be installed. Plugin may not work correctly.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.error('✅ Dependencies installed');
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('❌ Installation failed:', e.message);
|
||||
process.exit(1);
|
||||
}
|
||||
@@ -0,0 +1,477 @@
|
||||
#!/usr/bin/env bun
|
||||
import ts from 'typescript';
|
||||
import postcss from 'postcss';
|
||||
import { unified } from 'unified';
|
||||
import remarkParse from 'remark-parse';
|
||||
import remarkMdx from 'remark-mdx';
|
||||
import { visit } from 'unist-util-visit';
|
||||
import { parse as parse5Parse, parseFragment as parse5ParseFragment } from 'parse5';
|
||||
import { readFileSync, writeFileSync, statSync } from 'node:fs';
|
||||
import { execSync } from 'node:child_process';
|
||||
import { extname, basename, join } from 'node:path';
|
||||
|
||||
interface CliOptions {
|
||||
root: string;
|
||||
check: boolean;
|
||||
dryRun: boolean;
|
||||
verbose: boolean;
|
||||
}
|
||||
|
||||
function parseArgs(argv: string[]): CliOptions {
|
||||
let root = process.cwd();
|
||||
let check = false;
|
||||
let dryRun = false;
|
||||
let verbose = false;
|
||||
for (const arg of argv.slice(2)) {
|
||||
if (arg === '--check') check = true;
|
||||
else if (arg === '--dry-run') dryRun = true;
|
||||
else if (arg === '--verbose' || arg === '-v') verbose = true;
|
||||
else if (arg === '--help' || arg === '-h') {
|
||||
printHelp();
|
||||
process.exit(0);
|
||||
} else if (!arg.startsWith('-')) {
|
||||
root = arg;
|
||||
} else {
|
||||
console.error(`Unknown flag: ${arg}`);
|
||||
printHelp();
|
||||
process.exit(2);
|
||||
}
|
||||
}
|
||||
return { root, check, dryRun, verbose };
|
||||
}
|
||||
|
||||
function printHelp(): void {
|
||||
console.log(`Usage: bun scripts/strip-comments.ts [path] [flags]
|
||||
|
||||
Strips narrative comments from all git-tracked files using real parsers:
|
||||
JS/TS/JSX -> TypeScript compiler (ts.getLeadingCommentRanges)
|
||||
CSS/SCSS -> postcss + postcss-discard-comments
|
||||
MD/MDX -> unified + remark-parse + remark-mdx + remark-stringify
|
||||
HTML -> parse5 (parse, drop #comment nodes, serialize)
|
||||
shell/py -> line-based hash stripper (kept; no library worth its weight)
|
||||
|
||||
Build directives are preserved (shebangs, @ts-*, eslint-disable, biome-ignore,
|
||||
prettier-ignore, triple-slash references, webpack magic, /*! license keep).
|
||||
Files containing @strip-comments-keep in the first 4096 bytes are skipped.
|
||||
|
||||
Flags:
|
||||
--check Exit non-zero if any file would change. Doesn't write.
|
||||
--dry-run Print what would change without writing.
|
||||
--verbose Log each changed file.
|
||||
-h, --help Show this help.
|
||||
|
||||
After running, review the diff with \`git diff\`, then run
|
||||
\`npm run build-and-sync\` to confirm typecheck and build still pass.
|
||||
`);
|
||||
}
|
||||
|
||||
const SKIP_PATHS = new Set<string>([
|
||||
'package-lock.json',
|
||||
'bun.lock',
|
||||
'bun.lockb',
|
||||
'plugin/scripts/claude-mem',
|
||||
]);
|
||||
|
||||
const SKIP_BASENAMES = new Set<string>([
|
||||
'LICENSE',
|
||||
'COPYING',
|
||||
'NOTICE',
|
||||
]);
|
||||
|
||||
const BINARY_EXT = new Set<string>([
|
||||
'.svg', '.webp', '.woff', '.woff2', '.gif', '.png', '.jpg', '.jpeg', '.ico', '.pdf', '.zip',
|
||||
]);
|
||||
|
||||
const JS_LIKE_EXT = new Set<string>(['.ts', '.tsx', '.js', '.jsx', '.cjs', '.mjs']);
|
||||
const MD_EXT = new Set<string>(['.md', '.mdx']);
|
||||
const HTML_EXT = new Set<string>(['.html', '.htm']);
|
||||
const CSS_LIKE_EXT = new Set<string>(['.css', '.scss', '.less']);
|
||||
const HASH_LIKE_EXT = new Set<string>(['.sh', '.bash', '.zsh', '.py']);
|
||||
const HASH_LIKE_BASE = new Set<string>([
|
||||
'.gitignore', '.npmignore', '.dockerignore', '.gitattributes', '.npmrc', '.editorconfig',
|
||||
]);
|
||||
|
||||
const KEEP_MARKER = /@strip-comments-keep/;
|
||||
const NUL_BYTE = 0;
|
||||
|
||||
function isDirectiveJs(text: string): boolean {
|
||||
if (text.startsWith('///')) return true;
|
||||
if (text.startsWith('/*!')) return true;
|
||||
if (KEEP_MARKER.test(text)) return true;
|
||||
const inner = text
|
||||
.replace(/^\/\/\s*/, '')
|
||||
.replace(/^\/\*+\s*/, '')
|
||||
.replace(/\s*\*+\/$/, '')
|
||||
.trim();
|
||||
return /^(@ts-(?:ignore|expect-error|nocheck|check)\b|eslint-(?:disable|enable)|biome-ignore|prettier-ignore|@vitest-|c8\s+ignore|istanbul\s+ignore|@__PURE__|#__PURE__|webpack(?:ChunkName|Prefetch|Preload|Include|Exclude|Mode|Ignore))/.test(inner);
|
||||
}
|
||||
|
||||
function scriptKindFor(ext: string): ts.ScriptKind {
|
||||
switch (ext) {
|
||||
case '.tsx': return ts.ScriptKind.TSX;
|
||||
case '.jsx': return ts.ScriptKind.JSX;
|
||||
case '.js':
|
||||
case '.cjs':
|
||||
case '.mjs': return ts.ScriptKind.JS;
|
||||
default: return ts.ScriptKind.TS;
|
||||
}
|
||||
}
|
||||
|
||||
function parseDiagnosticsCount(sf: ts.SourceFile): number {
|
||||
return ((sf as unknown as { parseDiagnostics?: ts.Diagnostic[] }).parseDiagnostics ?? []).length;
|
||||
}
|
||||
|
||||
function stripJsLike(source: string, ext: string): string {
|
||||
const kind = scriptKindFor(ext);
|
||||
const sf = ts.createSourceFile('input', source, ts.ScriptTarget.Latest, true, kind);
|
||||
const beforeErrs = parseDiagnosticsCount(sf);
|
||||
|
||||
const seen = new Set<string>();
|
||||
const ranges: Array<[number, number]> = [];
|
||||
|
||||
function visitNode(node: ts.Node): void {
|
||||
const leading = ts.getLeadingCommentRanges(source, node.getFullStart()) || [];
|
||||
const trailing = ts.getTrailingCommentRanges(source, node.getEnd()) || [];
|
||||
for (const r of leading) addRange(r);
|
||||
for (const r of trailing) addRange(r);
|
||||
ts.forEachChild(node, visitNode);
|
||||
}
|
||||
function addRange(r: ts.CommentRange): void {
|
||||
const key = `${r.pos}-${r.end}`;
|
||||
if (seen.has(key)) return;
|
||||
seen.add(key);
|
||||
const text = source.slice(r.pos, r.end);
|
||||
if (isDirectiveJs(text)) return;
|
||||
ranges.push([r.pos, r.end]);
|
||||
}
|
||||
|
||||
visitNode(sf);
|
||||
const out = spliceRanges(source, ranges);
|
||||
|
||||
const after = ts.createSourceFile('check', out, ts.ScriptTarget.Latest, true, kind);
|
||||
const afterErrs = parseDiagnosticsCount(after);
|
||||
if (afterErrs > beforeErrs) {
|
||||
throw new Error(`strip introduced ${afterErrs - beforeErrs} new parse error(s); refusing to write`);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function collapseBlankLines(s: string): string {
|
||||
return s.replace(/(?:[ \t]*\n){3,}/g, '\n\n');
|
||||
}
|
||||
|
||||
function spliceRanges(source: string, ranges: Array<[number, number]>): string {
|
||||
ranges.sort((a, b) => a[0] - b[0]);
|
||||
let out = source;
|
||||
for (let i = ranges.length - 1; i >= 0; i--) {
|
||||
const [s, e] = ranges[i];
|
||||
let removeStart = s;
|
||||
let removeEnd = e;
|
||||
let lineStart = s;
|
||||
while (lineStart > 0 && (out[lineStart - 1] === ' ' || out[lineStart - 1] === '\t')) {
|
||||
lineStart--;
|
||||
}
|
||||
if (lineStart === 0 || out[lineStart - 1] === '\n') {
|
||||
removeStart = lineStart;
|
||||
if (out[removeEnd] === '\n') removeEnd++;
|
||||
}
|
||||
out = out.slice(0, removeStart) + out.slice(removeEnd);
|
||||
}
|
||||
return collapseBlankLines(out);
|
||||
}
|
||||
|
||||
function stripCss(source: string): string {
|
||||
const root = postcss.parse(source);
|
||||
const ranges: Array<[number, number]> = [];
|
||||
root.walkComments((node) => {
|
||||
const start = node.source?.start?.offset;
|
||||
const end = node.source?.end?.offset;
|
||||
if (typeof start !== 'number' || typeof end !== 'number') return;
|
||||
const raw = source.slice(start, end);
|
||||
if (raw.startsWith('/*!')) return;
|
||||
if (KEEP_MARKER.test(raw)) return;
|
||||
if (/\/\*\s*prettier-ignore/.test(raw)) return;
|
||||
ranges.push([start, end]);
|
||||
});
|
||||
return spliceRanges(source, ranges);
|
||||
}
|
||||
|
||||
const HTML_COMMENT_RE = /^<!--[\s\S]*-->$/;
|
||||
|
||||
function isMdxNarrativeComment(value: string): boolean {
|
||||
const trimmed = value.trim();
|
||||
if (/^\/\*[\s\S]*\*\/$/.test(trimmed)) return true;
|
||||
if (/^\/\/.*$/.test(trimmed)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
interface MdNode {
|
||||
type: string;
|
||||
value?: string;
|
||||
position?: { start?: { offset?: number }; end?: { offset?: number } };
|
||||
}
|
||||
|
||||
function stripMarkdown(source: string, isMdx: boolean): string {
|
||||
const processor = unified().use(remarkParse);
|
||||
if (isMdx) processor.use(remarkMdx);
|
||||
const tree = processor.parse(source);
|
||||
const ranges: Array<[number, number]> = [];
|
||||
visit(tree, (node) => {
|
||||
const n = node as MdNode;
|
||||
const start = n.position?.start?.offset;
|
||||
const end = n.position?.end?.offset;
|
||||
if (typeof start !== 'number' || typeof end !== 'number') return;
|
||||
if (n.type === 'html' && typeof n.value === 'string' && HTML_COMMENT_RE.test(n.value.trim())) {
|
||||
if (KEEP_MARKER.test(n.value)) return;
|
||||
ranges.push([start, end]);
|
||||
return;
|
||||
}
|
||||
if (isMdx && (n.type === 'mdxFlowExpression' || n.type === 'mdxTextExpression')) {
|
||||
const v = n.value ?? '';
|
||||
if (isMdxNarrativeComment(v) && !KEEP_MARKER.test(v)) {
|
||||
ranges.push([start, end]);
|
||||
}
|
||||
}
|
||||
});
|
||||
return spliceRanges(source, ranges);
|
||||
}
|
||||
|
||||
interface Parse5Node {
|
||||
nodeName: string;
|
||||
childNodes?: Parse5Node[];
|
||||
data?: string;
|
||||
sourceCodeLocation?: { startOffset?: number; endOffset?: number };
|
||||
}
|
||||
|
||||
function stripHtml(source: string, isFragment: boolean): string {
|
||||
const tree = (isFragment
|
||||
? parse5ParseFragment(source, { sourceCodeLocationInfo: true })
|
||||
: parse5Parse(source, { sourceCodeLocationInfo: true })) as unknown as Parse5Node;
|
||||
const ranges: Array<[number, number]> = [];
|
||||
collectHtmlCommentRanges(tree, source, ranges);
|
||||
return spliceRanges(source, ranges);
|
||||
}
|
||||
|
||||
function collectHtmlCommentRanges(node: Parse5Node, source: string, ranges: Array<[number, number]>): void {
|
||||
if (node.nodeName === '#comment') {
|
||||
const start = node.sourceCodeLocation?.startOffset;
|
||||
const end = node.sourceCodeLocation?.endOffset;
|
||||
if (typeof start === 'number' && typeof end === 'number') {
|
||||
const raw = source.slice(start, end);
|
||||
if (!KEEP_MARKER.test(raw)) {
|
||||
ranges.push([start, end]);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (node.childNodes) {
|
||||
for (const child of node.childNodes) {
|
||||
collectHtmlCommentRanges(child, source, ranges);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function stripHashComments(source: string, preserveShebang: boolean): string {
|
||||
const lines = source.split('\n');
|
||||
const out: string[] = [];
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
if (i === 0 && preserveShebang && line.startsWith('#!')) {
|
||||
out.push(line);
|
||||
continue;
|
||||
}
|
||||
const stripped = stripHashFromLine(line);
|
||||
if (stripped === '' && line.trim().startsWith('#')) {
|
||||
continue;
|
||||
}
|
||||
out.push(stripped);
|
||||
}
|
||||
return collapseBlankLines(out.join('\n'));
|
||||
}
|
||||
|
||||
function stripHashFromLine(line: string): string {
|
||||
let inSingle = false;
|
||||
let inDouble = false;
|
||||
let inBacktick = false;
|
||||
for (let i = 0; i < line.length; i++) {
|
||||
const c = line[i];
|
||||
if (c === '\\' && i + 1 < line.length) {
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (!inDouble && !inBacktick && c === "'") inSingle = !inSingle;
|
||||
else if (!inSingle && !inBacktick && c === '"') inDouble = !inDouble;
|
||||
else if (!inSingle && !inDouble && c === '`') inBacktick = !inBacktick;
|
||||
else if (!inSingle && !inDouble && !inBacktick && c === '#') {
|
||||
if (i === 0 || /\s/.test(line[i - 1])) {
|
||||
return line.slice(0, i).replace(/[ \t]+$/, '');
|
||||
}
|
||||
}
|
||||
}
|
||||
return line;
|
||||
}
|
||||
|
||||
interface Stats {
|
||||
changed: number;
|
||||
unchanged: number;
|
||||
skipped: number;
|
||||
bytesBefore: number;
|
||||
bytesAfter: number;
|
||||
errors: string[];
|
||||
changedFiles: string[];
|
||||
reformatRatioFlags: string[];
|
||||
}
|
||||
|
||||
function processFile(absPath: string, relPath: string, stats: Stats, opts: CliOptions): void {
|
||||
if (SKIP_PATHS.has(relPath)) {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
const base = basename(relPath);
|
||||
if (SKIP_BASENAMES.has(base)) {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
const ext = extname(relPath).toLowerCase();
|
||||
if (BINARY_EXT.has(ext)) {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
|
||||
let st;
|
||||
try {
|
||||
st = statSync(absPath);
|
||||
} catch {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
if (!st.isFile()) {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
|
||||
let raw: Buffer;
|
||||
try {
|
||||
raw = readFileSync(absPath);
|
||||
} catch {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
|
||||
if (raw.includes(NUL_BYTE)) {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
|
||||
const original = raw.toString('utf-8');
|
||||
if (KEEP_MARKER.test(original.slice(0, 4096))) {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
|
||||
let stripped: string;
|
||||
try {
|
||||
if (JS_LIKE_EXT.has(ext)) {
|
||||
stripped = stripJsLike(original, ext);
|
||||
} else if (CSS_LIKE_EXT.has(ext)) {
|
||||
stripped = stripCss(original);
|
||||
} else if (MD_EXT.has(ext)) {
|
||||
stripped = stripMarkdown(original, ext === '.mdx');
|
||||
} else if (HTML_EXT.has(ext)) {
|
||||
stripped = stripHtml(original, false);
|
||||
} else if (
|
||||
HASH_LIKE_EXT.has(ext) ||
|
||||
HASH_LIKE_BASE.has(base) ||
|
||||
base === 'Dockerfile' ||
|
||||
base.startsWith('Dockerfile.')
|
||||
) {
|
||||
stripped = stripHashComments(original, true);
|
||||
} else {
|
||||
stats.skipped++;
|
||||
return;
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
stats.errors.push(`${relPath}: ${(e as Error).message}`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (stripped === original) {
|
||||
stats.unchanged++;
|
||||
return;
|
||||
}
|
||||
|
||||
const removedBytes = original.length - stripped.length;
|
||||
if (removedBytes > 0) {
|
||||
const lineDiff = Math.abs(original.split('\n').length - stripped.split('\n').length);
|
||||
const removedFraction = removedBytes / Math.max(original.length, 1);
|
||||
if (lineDiff > 20 && removedFraction < 0.005) {
|
||||
stats.reformatRatioFlags.push(relPath);
|
||||
}
|
||||
}
|
||||
|
||||
stats.changed++;
|
||||
stats.bytesBefore += original.length;
|
||||
stats.bytesAfter += stripped.length;
|
||||
stats.changedFiles.push(relPath);
|
||||
|
||||
if (!opts.check && !opts.dryRun) {
|
||||
writeFileSync(absPath, stripped, 'utf-8');
|
||||
}
|
||||
if (opts.verbose || opts.dryRun) {
|
||||
console.log(`would change: ${relPath}`);
|
||||
}
|
||||
}
|
||||
|
||||
function main(): void {
|
||||
const opts = parseArgs(process.argv);
|
||||
|
||||
const stats: Stats = {
|
||||
changed: 0,
|
||||
unchanged: 0,
|
||||
skipped: 0,
|
||||
bytesBefore: 0,
|
||||
bytesAfter: 0,
|
||||
errors: [],
|
||||
changedFiles: [],
|
||||
reformatRatioFlags: [],
|
||||
};
|
||||
|
||||
let files: string[];
|
||||
try {
|
||||
files = execSync('git ls-files', { cwd: opts.root, encoding: 'utf-8' })
|
||||
.split('\n')
|
||||
.map((l) => l.trim())
|
||||
.filter((l) => l.length > 0);
|
||||
} catch (e) {
|
||||
console.error(`git ls-files failed in ${opts.root}: ${(e as Error).message}`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
for (const rel of files) {
|
||||
processFile(join(opts.root, rel), rel, stats, opts);
|
||||
}
|
||||
|
||||
const suffix = opts.check ? ' (check mode, no writes)' : opts.dryRun ? ' (dry-run, no writes)' : '';
|
||||
console.log(`Changed: ${stats.changed}${suffix}`);
|
||||
console.log(`Unchanged: ${stats.unchanged}`);
|
||||
console.log(`Skipped: ${stats.skipped}`);
|
||||
if (stats.changed > 0) {
|
||||
const saved = stats.bytesBefore - stats.bytesAfter;
|
||||
const pct = ((saved / stats.bytesBefore) * 100).toFixed(1);
|
||||
console.log(`Bytes: ${stats.bytesBefore} -> ${stats.bytesAfter} (-${saved}, -${pct}%)`);
|
||||
}
|
||||
if (stats.reformatRatioFlags.length > 0) {
|
||||
console.log(`Reformat-suspect (${stats.reformatRatioFlags.length}): library may be reformatting more than stripping`);
|
||||
for (const f of stats.reformatRatioFlags.slice(0, 10)) console.log(` ${f}`);
|
||||
}
|
||||
if (stats.errors.length > 0) {
|
||||
console.log(`Errors (${stats.errors.length}):`);
|
||||
for (const e of stats.errors.slice(0, 20)) {
|
||||
console.log(` ${e}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (stats.errors.length > 0) process.exit(1);
|
||||
if (opts.check && stats.changed > 0) process.exit(1);
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -1,10 +1,4 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Protected sync-marketplace script
|
||||
*
|
||||
* Prevents accidental rsync overwrite when installed plugin is on beta branch.
|
||||
* If on beta, the user should use the UI to update instead.
|
||||
*/
|
||||
|
||||
const { execSync } = require('child_process');
|
||||
const { existsSync, readFileSync } = require('fs');
|
||||
@@ -14,6 +8,13 @@ const os = require('os');
|
||||
const INSTALLED_PATH = path.join(os.homedir(), '.claude', 'plugins', 'marketplaces', 'thedotmack');
|
||||
const CACHE_BASE_PATH = path.join(os.homedir(), '.claude', 'plugins', 'cache', 'thedotmack', 'claude-mem');
|
||||
|
||||
// Reject obviously invalid ports before they reach http.request, which would
|
||||
// throw with a confusing error like "RangeError: Port should be > 0 and < 65536".
|
||||
function parseWorkerPort(value) {
|
||||
const port = Number.parseInt(String(value ?? ''), 10);
|
||||
return Number.isInteger(port) && port >= 1 && port <= 65535 ? port : null;
|
||||
}
|
||||
|
||||
function getCurrentBranch() {
|
||||
try {
|
||||
if (!existsSync(path.join(INSTALLED_PATH, '.git'))) {
|
||||
@@ -57,7 +58,6 @@ if (branch && branch !== 'main' && !isForce) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Get version from plugin.json
|
||||
function getPluginVersion() {
|
||||
try {
|
||||
const pluginJsonPath = path.join(__dirname, '..', 'plugin', '.claude-plugin', 'plugin.json');
|
||||
@@ -69,7 +69,60 @@ function getPluginVersion() {
|
||||
}
|
||||
}
|
||||
|
||||
// Normal rsync for main branch or fresh install
|
||||
function detectInstalledVersion(buildVersion) {
|
||||
const dataDir = process.env.CLAUDE_MEM_DATA_DIR || path.join(os.homedir(), '.claude-mem');
|
||||
const settingsPath = path.join(dataDir, 'settings.json');
|
||||
let port = parseWorkerPort(process.env.CLAUDE_MEM_WORKER_PORT);
|
||||
if (!port && existsSync(settingsPath)) {
|
||||
try {
|
||||
const s = JSON.parse(readFileSync(settingsPath, 'utf8'));
|
||||
const settingsPort = parseWorkerPort(s.CLAUDE_MEM_WORKER_PORT);
|
||||
if (settingsPort) port = settingsPort;
|
||||
} catch {}
|
||||
}
|
||||
if (!port) {
|
||||
const uid = typeof process.getuid === 'function' ? process.getuid() : 77;
|
||||
port = 37700 + (uid % 100);
|
||||
}
|
||||
let healthBody;
|
||||
try {
|
||||
healthBody = execSync(`curl -s --max-time 2 http://127.0.0.1:${port}/api/health`, {
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
}).toString().trim();
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (!healthBody) return null;
|
||||
let installedVersion;
|
||||
let installedPath;
|
||||
try {
|
||||
const j = JSON.parse(healthBody);
|
||||
installedVersion = j.version;
|
||||
installedPath = j.workerPath;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (!installedVersion || installedVersion === buildVersion) return null;
|
||||
return { installedVersion, installedPath };
|
||||
}
|
||||
|
||||
const installedMismatch = detectInstalledVersion(getPluginVersion());
|
||||
if (installedMismatch) {
|
||||
console.log('');
|
||||
console.log('\x1b[33m%s\x1b[0m', 'Version mismatch detected:');
|
||||
console.log(` Building: ${getPluginVersion()}`);
|
||||
console.log(` Installed: ${installedMismatch.installedVersion}`);
|
||||
if (installedMismatch.installedPath) console.log(` Worker path: ${installedMismatch.installedPath}`);
|
||||
console.log('');
|
||||
console.log('Claude Code is pinned to the installed version, so the worker loads from');
|
||||
console.log(`its cache dir. Mirroring this build into the installed-version cache so the`);
|
||||
console.log('worker restart picks up new code without a Claude Code session restart.');
|
||||
console.log('');
|
||||
console.log('\x1b[36m%s\x1b[0m', `For a formal version bump, run \`claude plugin update thedotmack/claude-mem\``);
|
||||
console.log('\x1b[36m%s\x1b[0m', `and restart Claude Code so it loads the ${getPluginVersion()} cache dir.`);
|
||||
console.log('');
|
||||
}
|
||||
|
||||
console.log('Syncing to marketplace...');
|
||||
try {
|
||||
const rootDir = path.join(__dirname, '..');
|
||||
@@ -86,7 +139,6 @@ try {
|
||||
{ stdio: 'inherit' }
|
||||
);
|
||||
|
||||
// Sync to cache folder with version
|
||||
const version = getPluginVersion();
|
||||
const CACHE_VERSION_PATH = path.join(CACHE_BASE_PATH, version);
|
||||
|
||||
@@ -99,19 +151,41 @@ try {
|
||||
{ stdio: 'inherit' }
|
||||
);
|
||||
|
||||
// Install dependencies in cache directory so worker can resolve them
|
||||
console.log(`Running bun install in cache folder (version ${version})...`);
|
||||
execSync(`bun install`, { cwd: CACHE_VERSION_PATH, stdio: 'inherit' });
|
||||
|
||||
if (installedMismatch && installedMismatch.installedVersion !== version) {
|
||||
const INSTALLED_CACHE_PATH = path.join(CACHE_BASE_PATH, installedMismatch.installedVersion);
|
||||
console.log(`Mirroring to installed-version cache (${installedMismatch.installedVersion}) for hot reload...`);
|
||||
execSync(
|
||||
`rsync -av --delete --exclude=.git ${pluginGitignoreExcludes} plugin/ "${INSTALLED_CACHE_PATH}/"`,
|
||||
{ stdio: 'inherit' }
|
||||
);
|
||||
console.log(`Running bun install in installed-version cache (${installedMismatch.installedVersion})...`);
|
||||
execSync(`bun install`, { cwd: INSTALLED_CACHE_PATH, stdio: 'inherit' });
|
||||
}
|
||||
|
||||
console.log('\x1b[32m%s\x1b[0m', 'Sync complete!');
|
||||
|
||||
// Trigger worker restart after file sync
|
||||
console.log('\n🔄 Triggering worker restart...');
|
||||
const http = require('http');
|
||||
const os = require('os');
|
||||
// Use per-user port derivation (#1936)
|
||||
const dataDir = process.env.CLAUDE_MEM_DATA_DIR || path.join(os.homedir(), '.claude-mem');
|
||||
const settingsPath = path.join(dataDir, 'settings.json');
|
||||
let settingsPort = null;
|
||||
if (existsSync(settingsPath)) {
|
||||
try {
|
||||
const settings = JSON.parse(readFileSync(settingsPath, 'utf8'));
|
||||
settingsPort = parseWorkerPort(settings.CLAUDE_MEM_WORKER_PORT);
|
||||
} catch {
|
||||
// fall through to env / default
|
||||
}
|
||||
}
|
||||
const uid = typeof process.getuid === 'function' ? process.getuid() : 77;
|
||||
const workerPort = parseInt(process.env.CLAUDE_MEM_WORKER_PORT || String(37700 + (uid % 100)), 10);
|
||||
const defaultPort = 37700 + (uid % 100);
|
||||
const workerPort =
|
||||
parseWorkerPort(process.env.CLAUDE_MEM_WORKER_PORT) ??
|
||||
settingsPort ??
|
||||
defaultPort;
|
||||
const req = http.request({
|
||||
hostname: '127.0.0.1',
|
||||
port: workerPort,
|
||||
@@ -120,17 +194,17 @@ try {
|
||||
timeout: 2000
|
||||
}, (res) => {
|
||||
if (res.statusCode === 200) {
|
||||
console.log('\x1b[32m%s\x1b[0m', '✓ Worker restart triggered');
|
||||
console.log('\x1b[32m%s\x1b[0m', `✓ Worker restart triggered on port ${workerPort}`);
|
||||
} else {
|
||||
console.log('\x1b[33m%s\x1b[0m', `ℹ Worker restart returned status ${res.statusCode}`);
|
||||
console.log('\x1b[33m%s\x1b[0m', `ℹ Worker restart on port ${workerPort} returned status ${res.statusCode}`);
|
||||
}
|
||||
});
|
||||
req.on('error', () => {
|
||||
console.log('\x1b[33m%s\x1b[0m', 'ℹ Worker not running, will start on next hook');
|
||||
console.log('\x1b[33m%s\x1b[0m', `ℹ No worker reachable on port ${workerPort}; the next worker:restart step will start one.`);
|
||||
});
|
||||
req.on('timeout', () => {
|
||||
req.destroy();
|
||||
console.log('\x1b[33m%s\x1b[0m', 'ℹ Worker restart timed out');
|
||||
console.log('\x1b[33m%s\x1b[0m', `ℹ Worker restart on port ${workerPort} timed out`);
|
||||
});
|
||||
req.end();
|
||||
|
||||
|
||||
@@ -1,21 +1,17 @@
|
||||
#!/bin/bash
|
||||
|
||||
# sync-to-marketplace.sh
|
||||
# Syncs the plugin folder to the Claude marketplace location
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
NC='\033[0m'
|
||||
|
||||
# Configuration
|
||||
SOURCE_DIR="plugin/"
|
||||
# Resolve SOURCE_DIR relative to this script so it works regardless of cwd.
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
SOURCE_DIR="$SCRIPT_DIR/../plugin/"
|
||||
DEST_DIR="$HOME/.claude/plugins/marketplaces/thedotmack/plugin/"
|
||||
|
||||
# Function to print colored output
|
||||
print_status() {
|
||||
echo -e "${GREEN}[INFO]${NC} $1"
|
||||
}
|
||||
@@ -28,13 +24,11 @@ print_error() {
|
||||
echo -e "${RED}[ERROR]${NC} $1"
|
||||
}
|
||||
|
||||
# Check if source directory exists
|
||||
if [ ! -d "$SOURCE_DIR" ]; then
|
||||
print_error "Source directory '$SOURCE_DIR' does not exist!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Create destination directory if it doesn't exist
|
||||
if [ ! -d "$DEST_DIR" ]; then
|
||||
print_warning "Destination directory '$DEST_DIR' does not exist. Creating it..."
|
||||
mkdir -p "$DEST_DIR"
|
||||
@@ -44,14 +38,12 @@ print_status "Syncing plugin folder to marketplace..."
|
||||
print_status "Source: $SOURCE_DIR"
|
||||
print_status "Destination: $DEST_DIR"
|
||||
|
||||
# Show what would be synced (dry run first)
|
||||
if [ "$1" = "--dry-run" ] || [ "$1" = "-n" ]; then
|
||||
print_status "Dry run - showing what would be synced:"
|
||||
rsync -av --delete --dry-run "$SOURCE_DIR" "$DEST_DIR"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Perform the actual sync
|
||||
if rsync -av --delete "$SOURCE_DIR" "$DEST_DIR"; then
|
||||
print_status "✅ Plugin folder synced successfully!"
|
||||
else
|
||||
@@ -59,7 +51,6 @@ else
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Show summary
|
||||
echo ""
|
||||
print_status "Sync complete. Files are now synchronized."
|
||||
print_status "You can run '$0 --dry-run' to preview changes before syncing."
|
||||
@@ -1,167 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Test script for TranscriptParser
|
||||
* Validates data extraction from Claude Code transcript JSONL files
|
||||
*
|
||||
* Usage: npx tsx scripts/test-transcript-parser.ts <path-to-transcript.jsonl>
|
||||
*/
|
||||
|
||||
import { TranscriptParser } from '../src/utils/transcript-parser.js';
|
||||
import { existsSync } from 'fs';
|
||||
import { resolve } from 'path';
|
||||
|
||||
function formatTokens(num: number): string {
|
||||
return num.toLocaleString();
|
||||
}
|
||||
|
||||
function formatPercentage(num: number): string {
|
||||
return `${(num * 100).toFixed(2)}%`;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const args = process.argv.slice(2);
|
||||
|
||||
if (args.length === 0) {
|
||||
console.error('Usage: npx tsx scripts/test-transcript-parser.ts <path-to-transcript.jsonl>');
|
||||
console.error('\nExample: npx tsx scripts/test-transcript-parser.ts ~/.cache/claude-code/transcripts/latest.jsonl');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const transcriptPath = resolve(args[0]);
|
||||
|
||||
if (!existsSync(transcriptPath)) {
|
||||
console.error(`Error: Transcript file not found: ${transcriptPath}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log(`\n🔍 Parsing transcript: ${transcriptPath}\n`);
|
||||
|
||||
try {
|
||||
const parser = new TranscriptParser(transcriptPath);
|
||||
|
||||
// Get parse statistics
|
||||
const stats = parser.getParseStats();
|
||||
|
||||
console.log('📊 Parse Statistics:');
|
||||
console.log('─'.repeat(60));
|
||||
console.log(`Total lines: ${stats.totalLines}`);
|
||||
console.log(`Parsed entries: ${stats.parsedEntries}`);
|
||||
console.log(`Failed lines: ${stats.failedLines}`);
|
||||
console.log(`Failure rate: ${formatPercentage(stats.failureRate)}`);
|
||||
console.log();
|
||||
|
||||
console.log('📋 Entries by Type:');
|
||||
console.log('─'.repeat(60));
|
||||
for (const [type, count] of Object.entries(stats.entriesByType)) {
|
||||
console.log(` ${type.padEnd(20)} ${count}`);
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Show parse errors if any
|
||||
if (stats.failedLines > 0) {
|
||||
console.log('❌ Parse Errors:');
|
||||
console.log('─'.repeat(60));
|
||||
const errors = parser.getParseErrors();
|
||||
errors.slice(0, 5).forEach(err => {
|
||||
console.log(` Line ${err.lineNumber}: ${err.error}`);
|
||||
});
|
||||
if (errors.length > 5) {
|
||||
console.log(` ... and ${errors.length - 5} more errors`);
|
||||
}
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Test data extraction methods
|
||||
console.log('💬 Message Extraction:');
|
||||
console.log('─'.repeat(60));
|
||||
|
||||
const lastUserMessage = parser.getLastUserMessage();
|
||||
console.log(`Last user message: ${lastUserMessage ? `"${lastUserMessage.substring(0, 100)}..."` : '(none)'}`);
|
||||
console.log();
|
||||
|
||||
const lastAssistantMessage = parser.getLastAssistantMessage();
|
||||
console.log(`Last assistant message: ${lastAssistantMessage ? `"${lastAssistantMessage.substring(0, 100)}..."` : '(none)'}`);
|
||||
console.log();
|
||||
|
||||
// Token usage
|
||||
const tokenUsage = parser.getTotalTokenUsage();
|
||||
console.log('💰 Token Usage:');
|
||||
console.log('─'.repeat(60));
|
||||
console.log(`Input tokens: ${formatTokens(tokenUsage.inputTokens)}`);
|
||||
console.log(`Output tokens: ${formatTokens(tokenUsage.outputTokens)}`);
|
||||
console.log(`Cache creation tokens: ${formatTokens(tokenUsage.cacheCreationTokens)}`);
|
||||
console.log(`Cache read tokens: ${formatTokens(tokenUsage.cacheReadTokens)}`);
|
||||
console.log(`Total tokens: ${formatTokens(tokenUsage.inputTokens + tokenUsage.outputTokens)}`);
|
||||
console.log();
|
||||
|
||||
// Tool use history
|
||||
const toolUses = parser.getToolUseHistory();
|
||||
console.log('🔧 Tool Use History:');
|
||||
console.log('─'.repeat(60));
|
||||
if (toolUses.length > 0) {
|
||||
console.log(`Total tool uses: ${toolUses.length}\n`);
|
||||
|
||||
// Group by tool name
|
||||
const toolCounts = toolUses.reduce((acc, tool) => {
|
||||
acc[tool.name] = (acc[tool.name] || 0) + 1;
|
||||
return acc;
|
||||
}, {} as Record<string, number>);
|
||||
|
||||
console.log('Tools used:');
|
||||
for (const [name, count] of Object.entries(toolCounts).sort((a, b) => b[1] - a[1])) {
|
||||
console.log(` ${name.padEnd(30)} ${count}x`);
|
||||
}
|
||||
} else {
|
||||
console.log('(no tool uses found)');
|
||||
}
|
||||
console.log();
|
||||
|
||||
// System entries
|
||||
const systemEntries = parser.getSystemEntries();
|
||||
if (systemEntries.length > 0) {
|
||||
console.log('⚠️ System Entries:');
|
||||
console.log('─'.repeat(60));
|
||||
console.log(`Found ${systemEntries.length} system entries`);
|
||||
systemEntries.slice(0, 3).forEach(entry => {
|
||||
console.log(` [${entry.level || 'info'}] ${entry.content.substring(0, 80)}...`);
|
||||
});
|
||||
if (systemEntries.length > 3) {
|
||||
console.log(` ... and ${systemEntries.length - 3} more`);
|
||||
}
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Summary entries
|
||||
const summaryEntries = parser.getSummaryEntries();
|
||||
if (summaryEntries.length > 0) {
|
||||
console.log('📝 Summary Entries:');
|
||||
console.log('─'.repeat(60));
|
||||
console.log(`Found ${summaryEntries.length} summary entries`);
|
||||
summaryEntries.forEach((entry, i) => {
|
||||
console.log(`\nSummary ${i + 1}:`);
|
||||
console.log(entry.summary.substring(0, 200) + '...');
|
||||
});
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Queue operations
|
||||
const queueOps = parser.getQueueOperationEntries();
|
||||
if (queueOps.length > 0) {
|
||||
console.log('🔄 Queue Operations:');
|
||||
console.log('─'.repeat(60));
|
||||
const enqueues = queueOps.filter(op => op.operation === 'enqueue').length;
|
||||
const dequeues = queueOps.filter(op => op.operation === 'dequeue').length;
|
||||
console.log(`Enqueue operations: ${enqueues}`);
|
||||
console.log(`Dequeue operations: ${dequeues}`);
|
||||
console.log();
|
||||
}
|
||||
|
||||
console.log('✅ Validation complete!\n');
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Error parsing transcript:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -1,209 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Transcript to Markdown - Complete 1:1 representation
|
||||
* Shows ALL available context data from a Claude Code transcript
|
||||
*/
|
||||
|
||||
import { TranscriptParser } from '../src/utils/transcript-parser.js';
|
||||
import type { UserTranscriptEntry, AssistantTranscriptEntry, ToolResultContent } from '../types/transcript.js';
|
||||
import { writeFileSync } from 'fs';
|
||||
import { basename } from 'path';
|
||||
|
||||
const transcriptPath = process.argv[2];
|
||||
const maxTurns = process.argv[3] ? parseInt(process.argv[3]) : 20;
|
||||
|
||||
if (!transcriptPath) {
|
||||
console.error('Usage: tsx scripts/transcript-to-markdown.ts <path-to-transcript.jsonl> [max-turns]');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Truncate string to max length, adding ellipsis if needed
|
||||
*/
|
||||
function truncate(str: string, maxLen: number = 500): string {
|
||||
if (str.length <= maxLen) return str;
|
||||
return str.substring(0, maxLen) + '\n... [truncated]';
|
||||
}
|
||||
|
||||
/**
|
||||
* Format tool result content for display
|
||||
*/
|
||||
function formatToolResult(result: ToolResultContent): string {
|
||||
if (typeof result.content === 'string') {
|
||||
// Try to parse as JSON for better formatting
|
||||
try {
|
||||
const parsed = JSON.parse(result.content);
|
||||
return JSON.stringify(parsed, null, 2);
|
||||
} catch {
|
||||
return truncate(result.content);
|
||||
}
|
||||
}
|
||||
|
||||
if (Array.isArray(result.content)) {
|
||||
// Handle array of content items - extract text and parse if JSON
|
||||
const formatted = result.content.map((item: any) => {
|
||||
if (item.type === 'text' && item.text) {
|
||||
try {
|
||||
const parsed = JSON.parse(item.text);
|
||||
return JSON.stringify(parsed, null, 2);
|
||||
} catch {
|
||||
return item.text;
|
||||
}
|
||||
}
|
||||
return JSON.stringify(item, null, 2);
|
||||
}).join('\n\n');
|
||||
|
||||
return formatted;
|
||||
}
|
||||
|
||||
return '[unknown result type]';
|
||||
}
|
||||
|
||||
const parser = new TranscriptParser(transcriptPath);
|
||||
const entries = parser.getAllEntries();
|
||||
const stats = parser.getParseStats();
|
||||
|
||||
let output = `# Transcript: ${basename(transcriptPath)}\n\n`;
|
||||
output += `**Generated:** ${new Date().toLocaleString()}\n`;
|
||||
output += `**Total Entries:** ${stats.parsedEntries}\n`;
|
||||
output += `**Entry Types:** ${JSON.stringify(stats.entriesByType, null, 2)}\n`;
|
||||
output += `**Showing:** First ${maxTurns} conversation turns\n\n`;
|
||||
|
||||
output += `---\n\n`;
|
||||
|
||||
let turnNumber = 0;
|
||||
let inTurn = false;
|
||||
|
||||
for (const entry of entries) {
|
||||
// Skip summary and file-history-snapshot entries
|
||||
if (entry.type === 'summary' || entry.type === 'file-history-snapshot') continue;
|
||||
|
||||
// USER MESSAGE
|
||||
if (entry.type === 'user') {
|
||||
const userEntry = entry as UserTranscriptEntry;
|
||||
|
||||
turnNumber++;
|
||||
if (turnNumber > maxTurns) break;
|
||||
|
||||
inTurn = true;
|
||||
output += `## Turn ${turnNumber}\n\n`;
|
||||
output += `### 👤 User\n`;
|
||||
output += `**Timestamp:** ${userEntry.timestamp}\n`;
|
||||
output += `**UUID:** ${userEntry.uuid}\n`;
|
||||
output += `**Session ID:** ${userEntry.sessionId}\n`;
|
||||
output += `**CWD:** ${userEntry.cwd}\n\n`;
|
||||
|
||||
// Extract user message text
|
||||
if (typeof userEntry.message.content === 'string') {
|
||||
output += userEntry.message.content + '\n\n';
|
||||
} else if (Array.isArray(userEntry.message.content)) {
|
||||
const textBlocks = userEntry.message.content.filter((c) => c.type === 'text');
|
||||
if (textBlocks.length > 0) {
|
||||
const text = textBlocks.map((b: any) => b.text).join('\n');
|
||||
output += text + '\n\n';
|
||||
}
|
||||
|
||||
// Show ACTUAL tool results with their data
|
||||
const toolResults = userEntry.message.content.filter((c): c is ToolResultContent => c.type === 'tool_result');
|
||||
if (toolResults.length > 0) {
|
||||
output += `**Tool Results Submitted (${toolResults.length}):**\n\n`;
|
||||
for (const result of toolResults) {
|
||||
output += `- **Tool Use ID:** \`${result.tool_use_id}\`\n`;
|
||||
if (result.is_error) {
|
||||
output += ` **ERROR:**\n`;
|
||||
}
|
||||
output += ` \`\`\`json\n`;
|
||||
output += ` ${formatToolResult(result)}\n`;
|
||||
output += ` \`\`\`\n\n`;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ASSISTANT MESSAGE
|
||||
if (entry.type === 'assistant' && inTurn) {
|
||||
const assistantEntry = entry as AssistantTranscriptEntry;
|
||||
|
||||
output += `### 🤖 Assistant\n`;
|
||||
output += `**Timestamp:** ${assistantEntry.timestamp}\n`;
|
||||
output += `**UUID:** ${assistantEntry.uuid}\n`;
|
||||
output += `**Model:** ${assistantEntry.message.model}\n`;
|
||||
output += `**Stop Reason:** ${assistantEntry.message.stop_reason || 'N/A'}\n\n`;
|
||||
|
||||
if (!Array.isArray(assistantEntry.message.content)) {
|
||||
output += `*[No content]*\n\n`;
|
||||
continue;
|
||||
}
|
||||
|
||||
const content = assistantEntry.message.content;
|
||||
|
||||
// 1. Thinking blocks (show first, as they happen first in reasoning)
|
||||
const thinkingBlocks = content.filter((c) => c.type === 'thinking');
|
||||
if (thinkingBlocks.length > 0) {
|
||||
output += `**💭 Thinking:**\n\n`;
|
||||
for (const block of thinkingBlocks) {
|
||||
const thinking = (block as any).thinking;
|
||||
// Format thinking with proper line breaks and indentation
|
||||
const formattedThinking = thinking
|
||||
.split('\n')
|
||||
.map((line: string) => line.trimEnd())
|
||||
.join('\n');
|
||||
|
||||
output += '> ';
|
||||
output += formattedThinking.replace(/\n/g, '\n> ');
|
||||
output += '\n\n';
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Text responses
|
||||
const textBlocks = content.filter((c) => c.type === 'text');
|
||||
if (textBlocks.length > 0) {
|
||||
output += `**Response:**\n\n`;
|
||||
for (const block of textBlocks) {
|
||||
output += (block as any).text + '\n\n';
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Tool uses - show complete input
|
||||
const toolUseBlocks = content.filter((c) => c.type === 'tool_use');
|
||||
if (toolUseBlocks.length > 0) {
|
||||
output += `**🔧 Tools Used (${toolUseBlocks.length}):**\n\n`;
|
||||
for (const tool of toolUseBlocks) {
|
||||
const t = tool as any;
|
||||
output += `- **${t.name}** (ID: \`${t.id}\`)\n`;
|
||||
output += ` \`\`\`json\n`;
|
||||
output += ` ${JSON.stringify(t.input, null, 2)}\n`;
|
||||
output += ` \`\`\`\n\n`;
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Token usage
|
||||
if (assistantEntry.message.usage) {
|
||||
const usage = assistantEntry.message.usage;
|
||||
output += `**📊 Token Usage:**\n`;
|
||||
output += `- Input: ${usage.input_tokens || 0}\n`;
|
||||
output += `- Output: ${usage.output_tokens || 0}\n`;
|
||||
if (usage.cache_creation_input_tokens) {
|
||||
output += `- Cache creation: ${usage.cache_creation_input_tokens}\n`;
|
||||
}
|
||||
if (usage.cache_read_input_tokens) {
|
||||
output += `- Cache read: ${usage.cache_read_input_tokens}\n`;
|
||||
}
|
||||
output += '\n';
|
||||
}
|
||||
|
||||
output += `---\n\n`;
|
||||
inTurn = false;
|
||||
}
|
||||
}
|
||||
|
||||
if (turnNumber < (stats.entriesByType['user'] || 0)) {
|
||||
output += `\n*... ${(stats.entriesByType['user'] || 0) - turnNumber} more turns not shown*\n`;
|
||||
}
|
||||
|
||||
// Write output
|
||||
const outputPath = transcriptPath.replace('.jsonl', '-complete.md');
|
||||
writeFileSync(outputPath, output, 'utf-8');
|
||||
|
||||
console.log(`\nComplete transcript written to: ${outputPath}`);
|
||||
console.log(`Turns shown: ${Math.min(turnNumber, maxTurns)} of ${stats.entriesByType['user'] || 0}\n`);
|
||||
@@ -67,7 +67,6 @@ SUPPORTED LANGUAGES:
|
||||
|
||||
function printLanguages(): void {
|
||||
const LANGUAGE_NAMES: Record<string, string> = {
|
||||
// Tier 1 - No-brainers
|
||||
zh: "Chinese (Simplified)",
|
||||
ja: "Japanese",
|
||||
"pt-br": "Brazilian Portuguese",
|
||||
@@ -75,7 +74,6 @@ function printLanguages(): void {
|
||||
es: "Spanish",
|
||||
de: "German",
|
||||
fr: "French",
|
||||
// Tier 2 - Strong tech scenes
|
||||
he: "Hebrew",
|
||||
ar: "Arabic",
|
||||
ru: "Russian",
|
||||
@@ -84,7 +82,6 @@ function printLanguages(): void {
|
||||
nl: "Dutch",
|
||||
tr: "Turkish",
|
||||
uk: "Ukrainian",
|
||||
// Tier 3 - Emerging/Growing fast
|
||||
vi: "Vietnamese",
|
||||
id: "Indonesian",
|
||||
th: "Thai",
|
||||
@@ -93,14 +90,12 @@ function printLanguages(): void {
|
||||
ur: "Urdu",
|
||||
ro: "Romanian",
|
||||
sv: "Swedish",
|
||||
// Tier 4 - Why not
|
||||
it: "Italian",
|
||||
el: "Greek",
|
||||
hu: "Hungarian",
|
||||
fi: "Finnish",
|
||||
da: "Danish",
|
||||
no: "Norwegian",
|
||||
// Other supported
|
||||
bg: "Bulgarian",
|
||||
et: "Estonian",
|
||||
lt: "Lithuanian",
|
||||
@@ -134,7 +129,7 @@ function parseArgs(argv: string[]): CliArgs {
|
||||
};
|
||||
|
||||
const positional: string[] = [];
|
||||
let i = 2; // Skip node and script path
|
||||
let i = 2;
|
||||
|
||||
while (i < argv.length) {
|
||||
const arg = argv[i];
|
||||
@@ -219,7 +214,6 @@ async function main(): Promise<void> {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Validate language codes
|
||||
const invalidLangs = args.languages.filter(
|
||||
(lang) => !SUPPORTED_LANGUAGES.includes(lang.toLowerCase())
|
||||
);
|
||||
@@ -243,7 +237,6 @@ async function main(): Promise<void> {
|
||||
useExisting: args.useExisting,
|
||||
});
|
||||
|
||||
// Exit with error code if any translations failed
|
||||
if (result.failed > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
@@ -1,12 +1,6 @@
|
||||
/**
|
||||
* Example: Using readme-translator in build scripts
|
||||
*
|
||||
* These examples show how to integrate the translator into your build pipeline.
|
||||
*/
|
||||
|
||||
import { translateReadme, TranslationJobResult, SUPPORTED_LANGUAGES } from "./index.js";
|
||||
|
||||
// Example 1: Simple usage - translate to a few common languages
|
||||
async function translateToCommonLanguages(): Promise<void> {
|
||||
const result = await translateReadme({
|
||||
source: "./README.md",
|
||||
@@ -17,7 +11,6 @@ async function translateToCommonLanguages(): Promise<void> {
|
||||
console.log(`Translated to ${result.successful} languages`);
|
||||
}
|
||||
|
||||
// Example 2: Full i18n setup with custom output directory
|
||||
async function fullI18nSetup(): Promise<void> {
|
||||
const result = await translateReadme({
|
||||
source: "./README.md",
|
||||
@@ -30,7 +23,6 @@ async function fullI18nSetup(): Promise<void> {
|
||||
verbose: true,
|
||||
});
|
||||
|
||||
// Handle results programmatically
|
||||
for (const r of result.results) {
|
||||
if (!r.success) {
|
||||
console.error(`Failed to translate to ${r.language}: ${r.error}`);
|
||||
@@ -38,9 +30,6 @@ async function fullI18nSetup(): Promise<void> {
|
||||
}
|
||||
}
|
||||
|
||||
// Example 3: Build script integration with error handling
|
||||
// Note: If Claude Code is authenticated, no API key needed locally.
|
||||
// CI/CD environments will need ANTHROPIC_API_KEY set.
|
||||
async function buildScriptIntegration(): Promise<number> {
|
||||
try {
|
||||
const result = await translateReadme({
|
||||
@@ -50,7 +39,6 @@ async function buildScriptIntegration(): Promise<number> {
|
||||
verbose: process.env.CI !== "true", // Quiet in CI
|
||||
});
|
||||
|
||||
// Return exit code for build scripts
|
||||
return result.failed > 0 ? 1 : 0;
|
||||
} catch (error) {
|
||||
console.error("Translation failed:", error);
|
||||
@@ -58,7 +46,6 @@ async function buildScriptIntegration(): Promise<number> {
|
||||
}
|
||||
}
|
||||
|
||||
// Example 4: Batch translation of multiple READMEs
|
||||
async function batchTranslation(): Promise<void> {
|
||||
const readmes = [
|
||||
"./README.md",
|
||||
@@ -78,9 +65,7 @@ async function batchTranslation(): Promise<void> {
|
||||
}
|
||||
}
|
||||
|
||||
// Example 5: Custom output pattern for docs sites
|
||||
async function docsiteSetup(): Promise<void> {
|
||||
// For docusaurus/vitepress style: docs/README.es.md
|
||||
await translateReadme({
|
||||
source: "./README.md",
|
||||
languages: ["es", "fr", "de", "ja", "zh"],
|
||||
@@ -90,9 +75,7 @@ async function docsiteSetup(): Promise<void> {
|
||||
});
|
||||
}
|
||||
|
||||
// Example 6: Conditional translation in CI/CD
|
||||
async function cicdTranslation(): Promise<void> {
|
||||
// Only translate on main branch releases
|
||||
const isRelease = process.env.GITHUB_REF === "refs/heads/main";
|
||||
const isManualTrigger = process.env.GITHUB_EVENT_NAME === "workflow_dispatch";
|
||||
|
||||
@@ -109,7 +92,6 @@ async function cicdTranslation(): Promise<void> {
|
||||
verbose: true,
|
||||
});
|
||||
|
||||
// Write summary for GitHub Actions
|
||||
if (process.env.GITHUB_STEP_SUMMARY) {
|
||||
const summary = `
|
||||
## Translation Summary
|
||||
@@ -117,12 +99,10 @@ async function cicdTranslation(): Promise<void> {
|
||||
- ❌ Failed: ${result.failed}
|
||||
- 💰 Cost: $${result.totalCostUsd.toFixed(4)}
|
||||
`;
|
||||
// In real usage, write to GITHUB_STEP_SUMMARY
|
||||
console.log(summary);
|
||||
}
|
||||
}
|
||||
|
||||
// Run an example
|
||||
const example = process.argv[2];
|
||||
|
||||
switch (example) {
|
||||
|
||||
@@ -31,25 +31,15 @@ async function writeCache(cachePath: string, cache: TranslationCache): Promise<v
|
||||
}
|
||||
|
||||
export interface TranslationOptions {
|
||||
/** Source README file path */
|
||||
source: string;
|
||||
/** Target languages (e.g., ['es', 'fr', 'de', 'ja', 'zh']) */
|
||||
languages: string[];
|
||||
/** Output directory (defaults to same directory as source) */
|
||||
outputDir?: string;
|
||||
/** Output filename pattern (use {lang} placeholder, defaults to 'README.{lang}.md') */
|
||||
pattern?: string;
|
||||
/** Preserve code blocks without translation */
|
||||
preserveCode?: boolean;
|
||||
/** Model to use (defaults to 'sonnet') */
|
||||
model?: string;
|
||||
/** Maximum budget in USD for the entire translation job */
|
||||
maxBudgetUsd?: number;
|
||||
/** Verbose output */
|
||||
verbose?: boolean;
|
||||
/** Force re-translation even if cached */
|
||||
force?: boolean;
|
||||
/** Use existing translation file (if present) as a reference */
|
||||
useExisting?: boolean;
|
||||
}
|
||||
|
||||
@@ -59,7 +49,6 @@ export interface TranslationResult {
|
||||
success: boolean;
|
||||
error?: string;
|
||||
costUsd?: number;
|
||||
/** Whether this was served from cache */
|
||||
cached?: boolean;
|
||||
}
|
||||
|
||||
@@ -71,7 +60,6 @@ export interface TranslationJobResult {
|
||||
}
|
||||
|
||||
const LANGUAGE_NAMES: Record<string, string> = {
|
||||
// Tier 1 - No-brainers
|
||||
zh: "Chinese (Simplified)",
|
||||
ja: "Japanese",
|
||||
"pt-br": "Brazilian Portuguese",
|
||||
@@ -79,7 +67,6 @@ const LANGUAGE_NAMES: Record<string, string> = {
|
||||
es: "Spanish",
|
||||
de: "German",
|
||||
fr: "French",
|
||||
// Tier 2 - Strong tech scenes
|
||||
he: "Hebrew",
|
||||
ar: "Arabic",
|
||||
ru: "Russian",
|
||||
@@ -88,7 +75,6 @@ const LANGUAGE_NAMES: Record<string, string> = {
|
||||
nl: "Dutch",
|
||||
tr: "Turkish",
|
||||
uk: "Ukrainian",
|
||||
// Tier 3 - Emerging/Growing fast
|
||||
vi: "Vietnamese",
|
||||
id: "Indonesian",
|
||||
th: "Thai",
|
||||
@@ -97,14 +83,12 @@ const LANGUAGE_NAMES: Record<string, string> = {
|
||||
ur: "Urdu",
|
||||
ro: "Romanian",
|
||||
sv: "Swedish",
|
||||
// Tier 4 - Why not
|
||||
it: "Italian",
|
||||
el: "Greek",
|
||||
hu: "Hungarian",
|
||||
fi: "Finnish",
|
||||
da: "Danish",
|
||||
no: "Norwegian",
|
||||
// Other supported
|
||||
bg: "Bulgarian",
|
||||
et: "Estonian",
|
||||
lt: "Lithuanian",
|
||||
@@ -197,12 +181,10 @@ Always output only the translated content without any surrounding explanation.`,
|
||||
},
|
||||
});
|
||||
|
||||
// Progress spinner frames
|
||||
const spinnerFrames = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"];
|
||||
let spinnerIdx = 0;
|
||||
|
||||
for await (const message of stream) {
|
||||
// Handle streaming text deltas
|
||||
if (message.type === "stream_event") {
|
||||
const event = message.event as { type: string; delta?: { type: string; text?: string } };
|
||||
if (event.type === "content_block_delta" && event.delta?.type === "text_delta" && event.delta.text) {
|
||||
@@ -217,7 +199,6 @@ Always output only the translated content without any surrounding explanation.`,
|
||||
}
|
||||
}
|
||||
|
||||
// Handle full assistant messages (fallback)
|
||||
if (message.type === "assistant") {
|
||||
for (const block of message.message.content) {
|
||||
if (block.type === "text" && !translation) {
|
||||
@@ -231,7 +212,6 @@ Always output only the translated content without any surrounding explanation.`,
|
||||
const result = message as SDKResultMessage;
|
||||
if (result.subtype === "success") {
|
||||
costUsd = result.total_cost_usd;
|
||||
// Use the result text if we didn't get it from streaming
|
||||
if (!translation && result.result) {
|
||||
translation = result.result;
|
||||
charCount = translation.length;
|
||||
@@ -240,12 +220,10 @@ Always output only the translated content without any surrounding explanation.`,
|
||||
}
|
||||
}
|
||||
|
||||
// Clear the progress line
|
||||
if (options.verbose) {
|
||||
process.stdout.write("\r" + " ".repeat(60) + "\r");
|
||||
}
|
||||
|
||||
// Strip markdown code fences if Claude wrapped the output
|
||||
let cleaned = translation.trim();
|
||||
if (cleaned.startsWith("```markdown")) {
|
||||
cleaned = cleaned.slice("```markdown".length);
|
||||
@@ -278,18 +256,14 @@ export async function translateReadme(
|
||||
useExisting = false,
|
||||
} = options;
|
||||
|
||||
// Run all translations in parallel (up to 10 concurrent)
|
||||
const parallel = Math.min(languages.length, 10);
|
||||
|
||||
// Read source file
|
||||
const sourcePath = path.resolve(source);
|
||||
const content = await fs.readFile(sourcePath, "utf-8");
|
||||
|
||||
// Determine output directory
|
||||
const outDir = outputDir ? path.resolve(outputDir) : path.dirname(sourcePath);
|
||||
await fs.mkdir(outDir, { recursive: true });
|
||||
|
||||
// Compute content hash and load cache
|
||||
const sourceHash = hashContent(content);
|
||||
const cachePath = path.join(outDir, ".translation-cache.json");
|
||||
const cache = await readCache(cachePath);
|
||||
@@ -306,12 +280,10 @@ export async function translateReadme(
|
||||
console.log("");
|
||||
}
|
||||
|
||||
// Worker function for a single language
|
||||
async function translateLang(lang: string): Promise<TranslationResult> {
|
||||
const outputFilename = pattern.replace("{lang}", lang);
|
||||
const outputPath = path.join(outDir, outputFilename);
|
||||
|
||||
// Check cache (unless --force)
|
||||
if (!force && isHashMatch && cache?.translations[lang]) {
|
||||
const outputExists = await fs.access(outputPath).then(() => true).catch(() => false);
|
||||
if (outputExists) {
|
||||
@@ -354,13 +326,11 @@ export async function translateReadme(
|
||||
}
|
||||
}
|
||||
|
||||
// Run with concurrency limit
|
||||
async function runWithConcurrency<T>(items: T[], limit: number, fn: (item: T) => Promise<TranslationResult>): Promise<TranslationResult[]> {
|
||||
const results: TranslationResult[] = [];
|
||||
const executing = new Set<Promise<void>>();
|
||||
|
||||
for (const item of items) {
|
||||
// Check budget before starting new translation
|
||||
if (maxBudgetUsd && totalCostUsd >= maxBudgetUsd) {
|
||||
results.push({
|
||||
language: String(item),
|
||||
@@ -378,20 +348,17 @@ export async function translateReadme(
|
||||
}
|
||||
});
|
||||
|
||||
// Create a wrapped promise that removes itself when done
|
||||
const wrapped = p.finally(() => {
|
||||
executing.delete(wrapped);
|
||||
});
|
||||
|
||||
executing.add(wrapped);
|
||||
|
||||
// Wait for a slot to open up if we're at the limit
|
||||
if (executing.size >= limit) {
|
||||
await Promise.race(executing);
|
||||
}
|
||||
}
|
||||
|
||||
// Wait for all remaining translations to complete
|
||||
await Promise.all(executing);
|
||||
return results;
|
||||
}
|
||||
@@ -399,7 +366,6 @@ export async function translateReadme(
|
||||
const translationResults = await runWithConcurrency(languages, parallel, translateLang);
|
||||
results.push(...translationResults);
|
||||
|
||||
// Save updated cache
|
||||
const newCache: TranslationCache = {
|
||||
sourceHash,
|
||||
lastUpdated: new Date().toISOString(),
|
||||
@@ -432,5 +398,4 @@ export async function translateReadme(
|
||||
};
|
||||
}
|
||||
|
||||
// Export language codes for convenience
|
||||
export const SUPPORTED_LANGUAGES = Object.keys(LANGUAGE_NAMES);
|
||||
|
||||
@@ -1,96 +0,0 @@
|
||||
/**
|
||||
* Export/Import types for memory data
|
||||
*
|
||||
* These types represent the structure of exported memory data.
|
||||
* They are aligned with the actual database schema and include all fields
|
||||
* needed for complete data export and import operations.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Observation record as stored in the database and exported
|
||||
*/
|
||||
export interface ObservationRecord {
|
||||
id: number;
|
||||
memory_session_id: string;
|
||||
project: string;
|
||||
text: string | null;
|
||||
type: string;
|
||||
title: string;
|
||||
subtitle: string | null;
|
||||
facts: string | null;
|
||||
narrative: string | null;
|
||||
concepts: string | null;
|
||||
files_read: string | null;
|
||||
files_modified: string | null;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number | null;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* SDK Session record as stored in the database and exported
|
||||
*/
|
||||
export interface SdkSessionRecord {
|
||||
id: number;
|
||||
content_session_id: string;
|
||||
memory_session_id: string;
|
||||
project: string;
|
||||
user_prompt: string;
|
||||
started_at: string;
|
||||
started_at_epoch: number;
|
||||
completed_at: string | null;
|
||||
completed_at_epoch: number | null;
|
||||
status: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Session Summary record as stored in the database and exported
|
||||
*/
|
||||
export interface SessionSummaryRecord {
|
||||
id: number;
|
||||
memory_session_id: string;
|
||||
project: string;
|
||||
request: string | null;
|
||||
investigated: string | null;
|
||||
learned: string | null;
|
||||
completed: string | null;
|
||||
next_steps: string | null;
|
||||
files_read: string | null;
|
||||
files_edited: string | null;
|
||||
notes: string | null;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number | null;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* User Prompt record as stored in the database and exported
|
||||
*/
|
||||
export interface UserPromptRecord {
|
||||
id: number;
|
||||
content_session_id: string;
|
||||
prompt_number: number;
|
||||
prompt_text: string;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Complete export data structure
|
||||
*/
|
||||
export interface ExportData {
|
||||
exportedAt: string;
|
||||
exportedAtEpoch: number;
|
||||
query: string;
|
||||
project?: string;
|
||||
totalObservations: number;
|
||||
totalSessions: number;
|
||||
totalSummaries: number;
|
||||
totalPrompts: number;
|
||||
observations: ObservationRecord[];
|
||||
sessions: SdkSessionRecord[];
|
||||
summaries: SessionSummaryRecord[];
|
||||
prompts: UserPromptRecord[];
|
||||
}
|
||||
@@ -1,12 +1,5 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Validate Timestamp Logic
|
||||
*
|
||||
* This script validates that the backlog timestamp logic would work correctly
|
||||
* by checking pending messages and simulating what timestamps they would get.
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
@@ -30,7 +23,6 @@ function main() {
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Check for pending messages
|
||||
const pendingStats = db.query(`
|
||||
SELECT
|
||||
status,
|
||||
@@ -51,7 +43,6 @@ function main() {
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Get sample pending messages with their session info
|
||||
const pendingWithSessions = db.query(`
|
||||
SELECT
|
||||
pm.id,
|
||||
@@ -86,7 +77,6 @@ function main() {
|
||||
console.log(` Session started: ${formatTimestamp(msg.session_started)}`);
|
||||
console.log(` Project: ${msg.project}`);
|
||||
|
||||
// Validate logic
|
||||
const ageDays = Math.round((Date.now() - msg.msg_created) / (1000 * 60 * 60 * 24));
|
||||
|
||||
if (msg.msg_created < msg.session_started) {
|
||||
@@ -106,7 +96,7 @@ function main() {
|
||||
console.log('\nTimestamp Logic Validation:\n');
|
||||
console.log('✅ Code Flow:');
|
||||
console.log(' 1. SessionManager.yieldNextMessage() tracks earliestPendingTimestamp');
|
||||
console.log(' 2. SDKAgent captures originalTimestamp before processing');
|
||||
console.log(' 2. ClaudeProvider captures originalTimestamp before processing');
|
||||
console.log(' 3. processSDKResponse passes originalTimestamp to storeObservation/storeSummary');
|
||||
console.log(' 4. SessionStore uses overrideTimestampEpoch ?? Date.now()');
|
||||
console.log(' 5. earliestPendingTimestamp reset after batch completes\n');
|
||||
@@ -116,7 +106,6 @@ function main() {
|
||||
console.log(' - Backlog messages: get original created_at_epoch');
|
||||
console.log(' - Observations match their source message timestamps\n');
|
||||
|
||||
// Check for any sessions with stuck processing messages
|
||||
const stuckMessages = db.query(`
|
||||
SELECT
|
||||
session_db_id,
|
||||
|
||||
@@ -1,24 +1,15 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Verify Timestamp Fix
|
||||
*
|
||||
* This script verifies that the timestamp corruption has been properly fixed.
|
||||
* It checks for any remaining observations in the bad window that shouldn't be there.
|
||||
*/
|
||||
|
||||
import Database from 'bun:sqlite';
|
||||
import { resolve } from 'path';
|
||||
|
||||
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
|
||||
|
||||
// Bad window: Dec 24 19:45-20:31 (using actual epoch format from database)
|
||||
const BAD_WINDOW_START = 1766623500000; // Dec 24 19:45 PST
|
||||
const BAD_WINDOW_END = 1766626260000; // Dec 24 20:31 PST
|
||||
const BAD_WINDOW_START = 1766623500000;
|
||||
const BAD_WINDOW_END = 1766626260000;
|
||||
|
||||
// Original corruption window: Dec 16-22 (when sessions actually started)
|
||||
const ORIGINAL_WINDOW_START = 1765914000000; // Dec 16 00:00 PST
|
||||
const ORIGINAL_WINDOW_END = 1766613600000; // Dec 23 23:59 PST
|
||||
const ORIGINAL_WINDOW_START = 1765914000000;
|
||||
const ORIGINAL_WINDOW_END = 1766613600000;
|
||||
|
||||
interface Observation {
|
||||
id: number;
|
||||
@@ -46,7 +37,6 @@ function main() {
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
try {
|
||||
// Check 1: Observations still in bad window
|
||||
console.log('Check 1: Looking for observations still in bad window (Dec 24 19:45-20:31)...');
|
||||
const badWindowObs = db.query<Observation, []>(`
|
||||
SELECT id, memory_session_id, created_at_epoch, created_at, title
|
||||
@@ -67,7 +57,6 @@ function main() {
|
||||
}
|
||||
}
|
||||
|
||||
// Check 2: Observations now in original window
|
||||
console.log('Check 2: Counting observations in original window (Dec 17-20)...');
|
||||
const originalWindowObs = db.query<{ count: number }, []>(`
|
||||
SELECT COUNT(*) as count
|
||||
@@ -79,7 +68,6 @@ function main() {
|
||||
console.log(`Found ${originalWindowObs?.count || 0} observations in Dec 17-20 window`);
|
||||
console.log('(These should be the corrected observations)\n');
|
||||
|
||||
// Check 3: Session distribution
|
||||
console.log('Check 3: Session distribution of corrected observations...');
|
||||
const sessionDist = db.query<{ memory_session_id: string; count: number }, []>(`
|
||||
SELECT memory_session_id, COUNT(*) as count
|
||||
@@ -101,7 +89,6 @@ function main() {
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Check 4: Pending messages processed count
|
||||
console.log('Check 4: Verifying processed pending_messages...');
|
||||
const processedCount = db.query<{ count: number }, []>(`
|
||||
SELECT COUNT(*) as count
|
||||
@@ -113,7 +100,6 @@ function main() {
|
||||
|
||||
console.log(`${processedCount?.count || 0} pending messages were processed during bad window\n`);
|
||||
|
||||
// Summary
|
||||
console.log('═══════════════════════════════════════════════════════════════════════');
|
||||
console.log('VERIFICATION SUMMARY:');
|
||||
console.log('═══════════════════════════════════════════════════════════════════════\n');
|
||||
|
||||
@@ -1,8 +1,4 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Wipes the Chroma data directory so backfillAllProjects rebuilds it on next worker start.
|
||||
* Chroma is always rebuildable from SQLite — this is safe.
|
||||
*/
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const os = require('os');
|
||||
|
||||
Reference in New Issue
Block a user