The Agent Runtime is what actually happens between a user pressing Enter and your agent's answer appearing — the fixed pipeline, reasoning loop, and safety nets that run underneath every agent, regardless of which screen configured them.

Overview

The platform runs a fixed pipeline for every incoming message:

1. Duplicate check — if the agent is already generating a response for this conversation, the new trigger is dropped so two answers never race each other. A "generating" flag is set for the whole turn and cleared when the answer completes.

2. Input guardrails — the message runs through your configured prompt guardrails (PII detection, custom word filters, custom guardrail automations, and so on). If a guardrail blocks the message, the turn stops there and the blocked message is excluded from all future context; if a guardrail rewrites or masks the message, the agent sees the masked version.

3. Prerequisite actions — any prerequisite automations you configured run before the first agent turn in a session, and their outputs are stored so they don't run again for the same conversation.

4. Routing — if the target is an AI team, the team flow decides which agent or task runs; otherwise the agent's reasoning loop starts directly.

While all of this runs, the user's query is also indexed (when query indexing is enabled) so it shows up in analytics and query clustering.

Building the Prompt Each Turn

Every turn, the agent assembles its context fresh from four parts, in this order:

• System prompt — compiled from your Instructions fields (role, goal, background, instructions body), adjusted per model family, plus language/style preferences and, if configured, the output of a custom "additional context" automation appended at the end.

• Tool catalog — your enabled tools, task/topic tools, the knowledge tool (with the names of your attached knowledge sets), standard/global tools, skills tools, and slot-collection tools. Each task/topic is checked against its governance rules before being exposed.

• Conversation history — up to the most recent 2,000 messages of the conversation, oldest first. Messages blocked by guardrails and internal alert messages are excluded. When the agent is running a task, history is scoped to that task's messages only.

• A hidden context message — an invisible note giving the model the current date and time (in the workspace timezone) and any per-user context, so answers stay time-accurate. When prompt caching is enabled, only the date is included (not the time) so the cache stays valid through the day.

If the turn was started with an explicit task, the task text is placed as the very first user message; extra one-off instructions (for example, "now produce the final answer in this format") are appended as the last user message. If the assembled history would end on an assistant message, the runtime appends a small "Lets continue.." user message so the model always has something to respond to.

Managing Long Conversations

Three mechanisms keep long conversations inside the model's context window, all driven by the agent's context-management settings:

• Compaction (summarization) — when the conversation fills the model's context window past the Trigger point (default 70% of the context window), older messages are summarized into a running conversation summary by the Summarisation Model, and the summary replaces them in the prompt. The most recent context — the Preserved window (default 50%) — is kept verbatim. Summaries are checkpointed per conversation and per agent, so each new turn only summarizes what's new since the last checkpoint.

• Trimming — if trimming is on and the history passes the Trim Threshold (default 90% of the context window), older messages are collapsed into a single condensed block labelled as a trimmed conversation, cutting each message off once the budget is used and marking it "...(trimmed)".

• Dropping earlier turns' retrieved knowledge — an opt-in setting that removes the knowledge passages retrieved on previous turns from the history, so they aren't resent with every subsequent question.

You set Trigger, Preserved and Trim Threshold in Pre-processing > Compaction, each as a percentage of the context window or as a token count, and you can add exceptions for specific users or teams. See Preprocessing.

Tool results can additionally be converted to a compact token-efficient format (TOON) when that option is enabled, which shrinks large structured tool outputs without losing data.

Stopping Old Knowledge Passages from Resending

By default, when the agent searches its knowledge on a turn, the passages it retrieved stay in the conversation history and are sent again with every following question. On a long multi-turn conversation that inflates input tokens badly, because each turn carries every earlier turn's retrieved text as well as its own.

Context management has an opt-in setting — Remove past conversation knowledge chunks — that stops this. When it is on:

• What it drops: every knowledge retrieval that happened before the current user question — not just the immediately preceding turn. All earlier retrieved passages come out of the history.

• What it keeps: the retrieval done for the question being answered right now. The current turn's own knowledge is never stripped.

• It only removes knowledge retrievals. The actual questions and answers stay in history in full, so the agent still remembers what was discussed — it just no longer re-reads the source passages behind those earlier answers.

• Follow-up questions still get knowledge. Retrieval runs fresh on each turn. The one thing to know: if a follow-up depends on a detail that was only present in an earlier turn's raw passages and never made it into the earlier answer, the agent may need to retrieve again to recover it.

• It runs after compaction, so it works alongside summarization and trimming rather than instead of them — you can have all three on.

• It never fails a turn. If the step cannot run for any reason, the full history is kept and the answer still goes out.

Note: This setting is not exposed in the agent builder UI today — it is set on the agent's configuration through the API or the agent object, not from a settings screen. It sits under context management only; there is no equivalent control under pre-processing, so a value written anywhere else has no effect at runtime.

The Reasoning Loop and Its Safety Nets

Each turn is one cycle of a loop:

1. The runtime builds the prompt and calls the agent's execution model with the tool catalog.

2. The model's reply is validated and parsed into a thought, tool calls, and/or a final answer.

3. If there are tool calls, each tool runs, and its output is written back into the conversation as a tool result.

4. The loop re-triggers: the model is called again with the new tool results in context. This repeats until the model produces a final answer (or a stop condition hits).

Publishing citations is the one "tool" that doesn't restart the loop — when the model's final answer includes citation markers, the runtime publishes the citations and finishes the turn.

Three separate safety nets keep the loop bounded:

Tool Execution

For each tool call the model makes:

1. A status message ("thought") is shown to the user — either the model's own reasoning, or a configured static thought for that tool, falling back to "Handling your request...". You can suppress these with the disable-thought settings, or hide just the tool name.

2. If the tool has Require approval on, the run pauses: the user sees the tool's input (rendered as readable YAML, with connection identifiers stripped) and approves or denies. An "always allow" choice is remembered per user when you enabled that option. On deny, the tool is not executed — instead the model receives the result "User has denied to execute this tool." and continues from there.

3. Governance, rate-limit, and cache policies on the tool are evaluated; a cached result can be returned without executing, and a rate-limited call is rejected.

4. The tool executes — a connector action, workflow, MCP tool, or standard tool, each dispatched to its own handler.

5. The output is post-processed: it can pass through the tools guardrail wrapper, PII/redaction patterns are applied, and only then is it added to the conversation as the tool result the model sees.

6. Usage details (tool name, input, output) are saved with the message, which is what powers tool-use display in the session view and repetition detection.

Tool calls run both in parallel and one at a time, depending on the model response. When the model requests multiple tool calls in one reply and parallel tool-calling is enabled for that call, the runtime fires all of them simultaneously as independent executions and waits for every one to finish before re-invoking the model — one failing tool doesn't stop the others; each failure just increments the error counter and returns an error result to the model. When only a single tool is requested (or parallel calling is off), it runs synchronously inline.

If a tool fails, the conversation doesn't end. The error message becomes the tool's result and is fed back to the model as feedback, so the model can retry differently, pick another tool, or answer without it. If the model calls a tool name that doesn't exist, the runtime replies with feedback naming the invalid tool and telling the model to use only tools from its list or give a final answer.

Model Calls, Retries, and Reply Parsing

Model calls have built-in retries: up to 3 attempts by default. Retries automatically switch to non-streaming mode and drop provider service-tier hints to maximize the chance of success. Errors are categorized (rate limit, server busy, authentication, model not found, too many tokens, timeout/network, quota exceeded, server error, and so on) and recorded on the trace so you can see why a call failed. If every retry is exhausted, the run raises the error and the turn ends with a failure message. If an agent somehow has no model configured, a workspace-level default fallback model is used.

The raw model output is validated and cleaned before anything is shown:

• Thoughts are extracted from thinking tags (<think>, <thought>, <thinking>) or a Thought: prefix.

• The final answer is taken from a Final Answer: prefix, or — if there's no prefix and no tool call — the plain text itself becomes the answer.

• Follow-up suggestions (when enabled) are parsed from a trailing Followups: line and capped at 3.

• If the reply is malformed JSON (for structured output), an automatic JSON-repair pass tries to fix it; if the reply doesn't match the expected format at all, the model gets a "You didn't use the right format" correction and the loop retries.

The final answer also passes through your response guardrails before it's published.

How Citations Work at Runtime

When your agent retrieves knowledge or web results, each retrieved chunk gets a number. The model is instructed to mark the sentences it grounds on with $$[n]$$-style markers referencing those chunk numbers. When the final answer arrives, the runtime parses these markers, merges in any web-search chunks, and publishes citation entries (title, URL, icon, source content) attached to the message — that's what renders as clickable sources under the answer. Citation usage counts are also recorded against the knowledge chunks. Publishing citations never causes another reasoning-loop iteration. You can turn inline citations off with the disable-inline-citations setting.

Streaming and Live Updates

As the loop runs, the user sees thought/status messages per step, then the final answer streams in as it's generated (when the model supports streaming — models flagged as streaming-disabled fall back to a single response). The streaming channel sends a keep-alive event every 5 seconds whenever there's a quiet stretch, so the connection stays open through long tool runs. Users can stop generation mid-stream; the stream also carries distinct events for message chunks, message end, and completion. If a model call has to be retried after an error, the retry runs non-streamed and the finished text is delivered at once.

The runtime also notices if the user sends another message while it's still working: when the agent finishes reasoning, it compares the newest user message against the one it was answering; if the user sent something new in the meantime, the agent tells them "I noticed you sent another message while I was thinking. Let me include that in my response." and re-runs with the new message included, instead of answering the stale question.

Model Selection and Cost Tracking

Model choice is resolved fresh every turn, in this order of precedence:

1. If the current task/topic has its own execution model, that wins for turns inside the task.

2. If the turn is generating the final answer after knowledge retrieval, the Answer Generation Model (response-generation settings) is used.

3. On the first turn of a conversation (before any tool call), the topic/task selection model is used when configured.

4. Otherwise the agent's main execution model applies.

Streaming is also model-dependent: if the resolved model has streaming disabled, that turn is served non-streamed even if the agent streams elsewhere.

Every model call records input tokens, output tokens, prompt-cache read/write tokens, and their costs, and every step writes a trace entry (model, provider, connection, time taken, finish reason, success/failure, and error category on failures). Tool calls can carry a configured invocation cost that is resolved and added to usage. These per-step records roll up into agent- and user-level cost tracking, which is what spending limits and quota jobs enforce and what the metering dashboards display. Estimated cost is even checked before the call when quota limits are active.

Conversation, Task, and Topic State

The runtime keeps a small state record per conversation (current agent, current topic, how many times the agent has been triggered this turn, error count, generating flag) and a separate state record per task within the conversation (its own trigger count, its own current topic, which agent triggered it). That's why a task's history, loop budget, and topic selection are isolated from the rest of the conversation — each task is effectively its own thread inside the session.

Task/topic selection is itself a tool call. Depending on your Tool selection strategy, the model either calls one router tool (with the task name as input) or calls the specific task's own tool directly. When it fires:

1. The runtime looks up the task/topic by name for this agent and records it as the current topic on the conversation (or on the task state).

2. The task's instructions are fetched — including dynamically generated instructions when a dynamic-instructions automation is configured — and compiled into the tool result.

3. That result lands in the conversation, so from the next loop iteration the model is operating under the task's instructions, with the task's tools and (if set) the task's execution model.

Switching journeys mid-conversation uses the same mechanism with a "switch" variant of the tool. Every selection is recorded as a "Topic Selection" trace entry.