Context Management
Authoritative Definition
The runtime administration of prompt windows, encompassing the dynamic control of token usage, truncation strategies, memory eviction, and state updates.
Overview & Technical Description
Context Management operates during model execution to maintain the most relevant set of tokens while staying strictly within the models context window limits and adhering to cost and latency budgets. As interactions with an AI agent grow, the accumulated dialogue, retrieved documents, and tool outputs can easily exceed the physical constraints of the LLM. An effective context management system dynamically prioritizes information. It retains system instructions and critical working memory while selectively compressing, summarizing, or truncating older conversational turns and less relevant retrieved passages. By orchestrating this continuous flow of context, these systems ensure high reasoning quality without triggering out-of-memory errors or unbounded API costs, serving as the RAM controller for LLM applications.
Editorial Notes
Focuses on runtime operations rather than static document ingestion. Best practices involve partitioning the context window into distinct zones (e.g., permanent system instructions, rolling chat history, and ephemeral scratchpads).