context-compression

To compress long chat histories, apply extractive selection or abstractive summarization to preserve critical facts, names, and task-relevant details while satisfying a specific token budget calculation.

Reducing retrieved documents in RAG pipelines involves scoring information density by task relevance, selectively pruning lower-ranked chunks, and validating retained facts to maintain accurate Q&A reasoning.

Information-density scoring ranks sentences, paragraphs, or chunks by task relevance to prioritize what to keep, ensuring the context compression process preserves the most critical information for reasoning.

You can use extractive selection and abstractive summarization together to manage token budgets, combining key-point extraction and selective pruning to fit multi-source text into model context windows.

Avoid context compression when strict, verbatim recall of multi-turn agent interactions is required, as selective pruning and abstractive summarization inherently remove non-prioritized text to optimize token limits.

Calculate token budget by measuring the model context window, reserving tokens for prompts and outputs, and allocating the remaining budget across sources based on their information-density scores.