Context Compaction: Summarization for Long-Running Agents

A long-running agent accumulates context: initial instructions, tool calls, results, reasoning steps. Eventually this exceeds the context window and the agent either crashes or starts hallucinating. Context compaction — intelligently summarizing older context while preserving what matters — lets agents run for hours or days without degrading.

The Problem: Context Overflow

Start: 1K tokens
After 10 tool calls: 8K tokens
After 50 tool calls: 35K tokens
After 200 tool calls: 150K tokens → approaching limits
After 300 tool calls: 220K tokens → overflow → agent fails

Without compaction, long tasks either fail or force you to raise the model tier (expensive). With compaction, the agent runs indefinitely.

What to Preserve vs. Compress

Not all context is equally important. Build an importance model:

| Content | Preserve? | Compress to? | |---------|-----------|-------------| | Initial task and goals | Always | Keep verbatim | | Tool call errors (recent) | Recent | Include in summary | | Tool call results (recent) | Recent | Keep last 5 verbatim | | Old successful steps | Summarize | 1-2 sentence summary | | Intermediate reasoning | Compress | Key decisions only | | Accumulated facts | Always | Extract to fact list | | Dead-ends | Drop | Mention briefly |

Compaction Algorithm

interface ConversationSegment {
  id: string;
  messages: Anthropic.MessageParam[];
  tokenCount: number;
  importance: "critical" | "high" | "medium" | "low";
  summary?: string;
  keyFacts?: string[];
}

class ContextCompactor {
  private readonly MAX_TOKENS = 150_000;
  private readonly COMPACTION_THRESHOLD = 120_000; // compact when this is reached
  private readonly MIN_VERBATIM_RECENT = 10; // always keep last 10 messages verbatim

  async compact(segments: ConversationSegment[]): Promise<ConversationSegment[]> {
    const totalTokens = segments.reduce((sum, s) => sum + s.tokenCount, 0);
    if (totalTokens < this.COMPACTION_THRESHOLD) return segments;

    const recent = segments.slice(-this.MIN_VERBATIM_RECENT);
    const old = segments.slice(0, -this.MIN_VERBATIM_RECENT);
    const critical = old.filter((s) => s.importance === "critical");
    const compressible = old.filter((s) => s.importance !== "critical");

    // Compress the compressible segments
    const compressed = await this.compressSegments(compressible);

    // Rebuild: critical (verbatim) + compressed summary + recent (verbatim)
    return [...critical, compressed, ...recent];
  }

  private async compressSegments(segments: ConversationSegment[]): Promise<ConversationSegment> {
    const allMessages = segments.flatMap((s) => s.messages);
    const allFacts = segments.flatMap((s) => s.keyFacts ?? []);

    const response = await anthropic.messages.create({
      model: "claude-haiku-4-5-20251001",
      max_tokens: 1000,
      messages: [{
        role: "user",
        content: `Summarize this agent work history. Preserve: decisions made, facts learned, completed steps, open issues. 
Discard: intermediate reasoning, retried approaches, redundant observations.

Return JSON:
{
  "summary": "2-5 sentence narrative of what was done",
  "decisions": ["key decision 1", ...],
  "facts": ["fact 1", ...],
  "completedSteps": ["step 1", ...],
  "pendingIssues": ["issue 1", ...]
}

Messages:
${allMessages.map((m) => `${m.role}: ${JSON.stringify(m.content).slice(0, 500)}`).join("\n")}

Previously extracted facts: ${allFacts.join("; ")}`,
      }],
    });

    const { summary, decisions, facts, completedSteps, pendingIssues } =
      JSON.parse(response.content[0].text);

    const compressedContent = `[COMPRESSED HISTORY]
Summary: ${summary}
Key decisions: ${decisions.join("; ")}
Facts learned: ${facts.join("; ")}
Completed: ${completedSteps.join("; ")}
Pending: ${pendingIssues.join("; ")}`;

    return {
      id: crypto.randomUUID(),
      messages: [{
        role: "user" as const,
        content: compressedContent,
      }, {
        role: "assistant" as const,
        content: "Acknowledged. I have the above context.",
      }],
      tokenCount: estimateTokens(compressedContent),
      importance: "high",
      summary,
      keyFacts: facts,
    };
  }
}

Progressive Compaction in the Agent Loop

Trigger compaction automatically when approaching the limit:

class CompactionAwareAgentLoop {
  private compactor: ContextCompactor;
  private segments: ConversationSegment[] = [];
  private currentSegment: Anthropic.MessageParam[] = [];

  async run(goal: string, tools: Anthropic.Tool[]) {
    // Initial segment: task setup (always critical)
    this.segments.push({
      id: "initial",
      messages: [{ role: "user", content: goal }],
      tokenCount: estimateTokens(goal),
      importance: "critical",
    });

    let iterations = 0;
    while (iterations < 500) {
      iterations++;

      // Check if compaction needed
      const totalTokens = this.getTotalTokens();
      if (totalTokens > 100_000) {
        console.log(`Compacting at ${totalTokens} tokens...`);
        this.segments = await this.compactor.compact(this.segments);
        console.log(`After compaction: ${this.getTotalTokens()} tokens`);
      }

      // Build context from segments
      const messages = this.buildMessagesFromSegments();

      const response = await anthropic.messages.create({
        model: "claude-sonnet-4-6",
        max_tokens: 4096,
        tools,
        messages,
      });

      // Add response to current segment
      this.currentSegment.push({ role: "assistant", content: response.content });

      if (response.stop_reason === "end_turn") {
        await this.finalizeCurrentSegment();
        return this.extractFinalAnswer(response);
      }

      if (response.stop_reason === "tool_use") {
        const toolResults = await this.executeTools(response.content);
        this.currentSegment.push({ role: "user", content: toolResults });

        // Finalize segment after 5 tool calls to enable compaction
        if (this.currentSegment.length >= 10) {
          await this.finalizeCurrentSegment();
        }
      }
    }
  }

  private async finalizeCurrentSegment() {
    if (this.currentSegment.length === 0) return;

    const importance = this.determineImportance(this.currentSegment);
    const segment: ConversationSegment = {
      id: crypto.randomUUID(),
      messages: [...this.currentSegment],
      tokenCount: estimateTokens(JSON.stringify(this.currentSegment)),
      importance,
    };

    this.segments.push(segment);
    this.currentSegment = [];
  }

  private determineImportance(messages: Anthropic.MessageParam[]): ConversationSegment["importance"] {
    // Segments with errors or important tool results are high importance
    const hasErrors = messages.some((m) =>
      JSON.stringify(m.content).includes("Error:") || JSON.stringify(m.content).includes("failed")
    );
    const hasImportantResults = messages.some((m) =>
      JSON.stringify(m.content).length > 5000 // long results = likely important
    );

    if (hasErrors) return "high";
    if (hasImportantResults) return "medium";
    return "low";
  }

  private getTotalTokens(): number {
    return this.segments.reduce((sum, s) => sum + s.tokenCount, 0) +
           estimateTokens(JSON.stringify(this.currentSegment));
  }
}

Lossless vs. Lossy Compaction

Choose based on stakes:

type CompactionStrategy = "lossless" | "lossy" | "adaptive";

async function compactWithStrategy(
  messages: Anthropic.MessageParam[],
  strategy: CompactionStrategy
): Promise<string> {
  if (strategy === "lossless") {
    // Keep all facts, compress only narrative
    return compressPreservingAllFacts(messages);
  }

  if (strategy === "lossy") {
    // Aggressive compression: keep only key decisions and final states
    return aggressiveCompress(messages);
  }

  // Adaptive: measure information density per message, compress selectively
  const densities = await measureInformationDensity(messages);
  const lowDensityMessages = messages.filter((_, i) => densities[i] < 0.3);
  const highDensityMessages = messages.filter((_, i) => densities[i] >= 0.3);

  const compressedLow = await aggressiveCompress(lowDensityMessages);
  return `${compressedLow}\n[High-density messages preserved below]\n${formatMessages(highDensityMessages)}`;
}

What Claude Code Does Natively

Claude Code (the CLI tool at claude.ai/code) implements its own context compaction — when the context fills, it automatically summarizes older turns and continues the task. The same principles apply when building your own long-running agents.

FAQ

How much context can I recover after compaction? Lossless compaction retains ~90% of factual content in 20% of tokens. Lossy compaction retains ~70% in 10% of tokens. Test with your specific task type — some workflows are more compressible than others.

Should I compact eagerly or lazily? Lazy (compact when near limit) wastes less time on unnecessary compression. Eager (compact every N turns) gives more predictable performance. For production agents, lazy with a safety margin (compact at 80% of limit) is standard.

What if an important fact gets lost in compression? Use a "sacred facts" list — accumulate must-preserve facts explicitly and inject them at the start of every context window, outside the compacted section.

Does compaction work with streaming responses? Compaction happens between turns, not during streaming. Structure your agent loop to check for compaction need before the next LLM call.

How do I know compaction quality is adequate? Run your task with full context (where possible) and with compacted context. If outcomes diverge significantly, your compaction is too lossy. Adjust the compression prompt to preserve more detail.