feat(js/middleware/context-compression): Add LLM summarization

[ssbushi]

• LLM Summarization: Condenses older history into a summary message using a designated model while preserving the last N messages untouched.
• Custom prompt support
• Cost optimization: Skips the LLM summarization call if cheaper strategies save enough context.
• Adaptive overshoot scaling

[gemini-code-assist]

Code Review

This pull request introduces an LLM-based summarization strategy to the context compression middleware, allowing older conversation history to be condensed while preserving recent context. It also adds dynamic preservation window adjustments for budget overshoots and options to skip summarization if cheaper strategies yield sufficient savings. Feedback on these changes highlights three key areas for improvement: reordering the operations so summarization runs before truncation to prevent premature history loss, correcting a preservation window calculation to prevent aggressive message discarding, and refining text rendering by using replaceAll with a function callback to avoid special character replacement issues and using stringifyOutput to prevent double-escaping tool outputs.

[gemini-code-assist]

Using String.prototype.replace with a string replacement parameter can lead to unexpected behavior if the conversation text contains special replacement patterns (like $&, $1, etc.), which are common in code snippets or tool outputs. Additionally, replace only replaces the first occurrence. Using replaceAll with a function argument safely avoids interpreting these patterns and replaces all occurrences of the placeholder.

Suggested change

	const prompt = summaryPromptTemplate.replace(
	'{conversation}',
	conversationText
	);
	const prompt = summaryPromptTemplate.replaceAll(
	'{conversation}',
	() => conversationText
	);

[gemini-code-assist]

There are two significant issues in this block of code:

1. Architectural Order of Operations: Currently, message truncation (Step 5) is applied before summarization (Step 6). This causes the oldest messages to be permanently discarded before the summarization model has a chance to capture them, defeating the primary purpose of summarization (which is to preserve older history in a condensed form). Reordering these steps so that summarization runs first ensures maximum history preservation, with message truncation acting as a fallback or hard cap if summarization is skipped or fails.

2. Preservation Window Violation: There is a discrepancy between the commented logic in the tests (effectiveMaxMessages = max(2+1, 6 - (4 - 2)) = 4) and the actual implementation (Math.max(1, ...)). If effectiveMaxMessages is allowed to drop below adjustedPreserveRecent + 1 (when a truncation notice is inserted), the message truncation step can aggressively discard the very messages that were supposed to be preserved. Setting the lower bound to adjustedPreserveRecent + (insertTruncationNotice ? 1 : 0) ensures the preservation guarantees are strictly respected.

            // 5. Summarization
            if (summaryModelRef) {
              if (shouldSkipSummarization) {
                skippedSummary = true;
              } else {
                const sumResult = await applySummarization(
                  messages,
                  adjustedSummaryPreserveRecent
                );
                messages = sumResult.messages;
                isSummarized = sumResult.summarized;
              }
            }

            // 6. Message truncation (as fallback or hard cap)
            const effectiveMaxMessages = maxMessages
              ? Math.min(
                  maxMessages,
                  Math.max(
                    adjustedPreserveRecent + (insertTruncationNotice ? 1 : 0),
                    maxMessages - (basePreserveRecent - adjustedPreserveRecent)
                  )
                )
              : undefined;

            if (
              effectiveMaxMessages &&
              messages.length > effectiveMaxMessages
            ) {
              const msgResult = applyMessageTruncation(
                messages,
                effectiveMaxMessages
              );
              messages = msgResult.messages;
              noticeInserted = msgResult.noticeInserted;
            }

[gemini-code-assist]

Using JSON.stringify on p.toolResponse.output will double-escape and wrap string outputs in quotes, making the rendered conversation text less readable for the summarization model. Use the existing stringifyOutput helper instead, which safely handles both string and non-string outputs.

Suggested change

	if (p.toolResponse)
	return `[Tool response: ${p.toolResponse.name} → ${JSON.stringify(p.toolResponse.output)}]`;
	if (p.toolResponse)
	return `[Tool response: ${p.toolResponse.name} → ${stringifyOutput(p.toolResponse.output)}]`;