fix: prevent LLM from using <observation> tags in summary responses (#1345)

The SDK agent's conversation history is heavily biased toward
<observation> output. By the time a summarize prompt arrives, the
in-context conditioning can cause the LLM to respond with <observation>
tags instead of the expected <summary> tags. parseSummary() then returns
null and the summary is silently lost.

Two changes:
- Add explicit mode-switch instructions at the top of the summary prompt
  telling the LLM not to use <observation> tags and that only <summary>
  output will be accepted
- Add a warning log in parseSummary() when <observation> tags are found
  in a response that has no <summary> block, making the issue visible
  in logs instead of silently discarding

Fixes #1312

Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
This commit is contained in:
Umut Polat
2026-03-13 05:57:52 +03:00
committed by GitHub
parent 3651a34e96
commit 9dbf63f5d4
2 changed files with 10 additions and 1 deletions
+5
View File
@@ -120,6 +120,11 @@ export function parseSummary(text: string, sessionId?: number): ParsedSummary |
const summaryMatch = summaryRegex.exec(text);
if (!summaryMatch) {
// Log when the response contains <observation> instead of <summary>
// to help diagnose prompt conditioning issues (see #1312)
if (/<observation>/.test(text)) {
logger.warn('PARSER', 'Summary response contained <observation> tags instead of <summary> — prompt conditioning may need strengthening', { sessionId });
}
return null;
}
+5 -1
View File
@@ -130,7 +130,11 @@ export function buildSummaryPrompt(session: SDKSession, mode: ModeConfig): strin
return '';
})();
return `${mode.prompts.header_summary_checkpoint}
return `--- MODE SWITCH: PROGRESS SUMMARY ---
Do NOT output <observation> tags. This is a summary request, not an observation request.
Your response MUST use <summary> tags ONLY. Any <observation> output will be discarded.
${mode.prompts.header_summary_checkpoint}
${mode.prompts.summary_instruction}
${mode.prompts.summary_context_label}