xAI has introduced Context Compaction, a new feature for Grok-4.6 that enables developers to shrink extended conversation histories. This capability addresses scenarios where conversations exceed a few thousand tokens, leading to increased input costs and latency with each subsequent API call.
Context compaction transforms prior messages into a single opaque item. This item retains essential conversational elements such as system prompts, attached files, previous reasoning, and a compacted record of turns, while omitting verbose tool output and back-and-forth exchanges. The compacted item can then be passed verbatim in follow-up requests, allowing the model to continue the conversation as if the full history were present.
Key Points
- Context Compaction is a new feature for Grok-4.6.
- It shrinks long conversations into a single opaque compaction item.
- The compaction preserves salient state, including system prompts, attached files, and prior reasoning.
- It drops verbose tool output and back-and-forth exchanges.
- The compacted item is passed verbatim in subsequent requests.
- Compaction aims to lower input costs and reduce latency.
- It helps keep multi-hour agent loops within the model's context window.
Context
According to xAI, a typical pattern for using Context Compaction involves calling the Compaction API every N turns within an agent loop, or when the rendered context surpasses a chosen threshold. Developers can store the opaque encrypted_content blob in their own databases and pass it back to xAI's API unchanged.
Why It Matters
This feature offers developers a method to optimize the performance and cost-efficiency of applications built with Grok-4.6, particularly those involving extended, multi-turn interactions. It provides a mechanism to maintain conversational coherence without incurring the full cost and latency penalties of re-sending entire histories.
What To Do
- Note that compaction is effective when the conversation is large enough that input tokens are impacting cost or latency.
- Understand that compaction is for when the model still needs to remember prior turns.
- Confirm that the current conversation window fits within the model's context limit before compacting.
- Test the Compaction API by sending a conversation and using the returned item as the head of your next request.