Pre Processing tunes what happens to a user's query — and to the running conversation — before the agent generates an answer, from how the query is rephrased to how many knowledge chunks come back and how long conversations stay within budget.
Overview
The section's own description: "Optimize how the agent processes queries and retrieves knowledge for accurate responses and manages the context." From here you set the model that rephrases queries, how many knowledge chunks to retrieve, the model that re-ranks them, and Compaction, which keeps long conversations within the model's context budget. Query Rephrasing, Chunk Retrieval and Ranking save automatically. Compaction entries (Default and exceptions) open in their own editor — select Save there before you go back to Pre-processing. The section applies to agents and teams alike.
Query Rephrasing, Chunk Retrieval, and Ranking
Three settings govern how the agent finds the right knowledge:
Setting | What it does |
|---|---|
Query Rephrasing Model | Reformulates user queries to improve accuracy — rewrites the raw question into a form that retrieves better. |
Chunk Retrieval | Number of chunks to retrieve against one query. Default value: 50. Raise it for breadth, lower it for focus and speed. |
Ranking Model | Prioritizes retrieved chunks by reassessing their relevance and authority so the most relevant ones reach the answer step. |
How Query Rephrasing Behaves at Runtime
The rephrasing model sees the conversation history, the new message, and the user's context (profile fields like email or department), and it does more than reword:
Follow-ups become standalone questions — "for smoking?" after a policy question is retrieved as "what is the company policy for smoking?".
Chit-chat is short-circuited — greetings, or questions already answered in the conversation, get a direct reply with no knowledge retrieval at all.
Missing context is filled in — an incomplete question can be completed from the user's context (e.g., scoping "the leave policy" to the user's own department).
Rephrasing is skipped and the raw query used as-is when it's disabled on the agent, or when the agent runs in full agentic (task-executor) mode, where the agent forms its own knowledge queries. A custom rephraser prompt attached to the agent replaces the default behavior.
How Chunk Retrieval and Ranking Work Together
The vector search deliberately over-fetches — it pulls a few hundred candidate chunks by embedding similarity (optionally blended with keyword matching). The Ranking Model then re-scores those candidates against the query in batches and keeps only the top Chunk Retrieval count (default 50), which is what reaches the answer step. So Chunk Retrieval is the keep count after reranking, not the width of the initial search.
The ranking model can be a dedicated reranker, an LLM acting as a judge, or a custom reranking automation. If reranking is disabled on the agent, chunks pass through in raw similarity order instead; if a reranking call errors, the agent falls back to similarity order rather than failing the answer.
What Is Compaction and When Should I Use It?
Compaction compresses older conversation turns so long chats stay fast and within the model's context window. The settings card explains the trade-off: without trimming, "every call replays the full chat history — costs scale with conversation length and threads eventually hit the model's context window." Turning Compaction on gives you lower token spend, longer conversations, and faster responses by summarizing older turns.
Enable it for agents that hold long, multi-turn conversations; you can leave it off for short, transactional agents.
Configuring When Compaction Kicks In
With Compaction on, you edit a Default entry (and can Add exception entries for specific audiences). Each entry has:
Field | Description |
|---|---|
Trim Threshold | An "emergency trim when limit breached" level, set as a percentage or a token count. |
Trigger | The point at which compaction starts summarizing. |
Preserved | How much recent context to keep verbatim. |
Summarisation Model | Model used to generate summaries of the older turns. |
The defaults trigger compaction well before the limit and keep the most recent turns intact. Add an exception when one audience (say, a particular team) needs different thresholds than everyone else.