Agent Tools

What an agent can do during a turn: built-in tools, your own webhooks, tools your client runs, knowledge search with citations, and files that arrive with a message.

Tools are part of an agent's configuration, set with the calls on Agents. Every field: API reference.

Tools

Tools are declared on the agent version in tools. Three kinds exist, plus connectors, which bring in whatever an MCP server you run offers.

Some tool types are added to a run automatically and cannot be declared: backbone.tool_search, backbone.tool_call, backbone.skill_read, backbone.skills_apply, backbone.deliver and backbone.view_image. Saving an agent or agent version that adds one is refused with 400 NON_AUTHORABLE_TOOL, naming the type and why. An agent that already stores one keeps saving and rolling back.

Built-in tools

file_search runs inside 2kw.ai. The model never chooses which knowledge base to search beyond what you configured.

Web tools

Two built-in tools reach the public web. Both take an optional config.allowedDomains list of up to ten hostnames; a hostname also allows its subdomains, and an empty list allows any public site. An agent carries at most one of each.

{ "type": "web_search", "config": { "allowedDomains": ["europa.eu", "bund.de"] } }

web_search takes a query of up to 400 characters and an optional max_results (default 8; a value outside 1 to 10 is moved to the nearest bound). Each hit carries a title, URL, snippet and site, so the model can read a result with web_fetch next. An agent can search at most 10 times in one run. Each search counts once on the usage rate card. The in-app API reference lists every field of the envelope.

Search results are untrusted web content: the tool tells the model to treat them as information, never as instructions.

When a search fails, the tool answers with one of these error codes so the model can recover:

web_fetch reads only an address that already appeared in the conversation: one the user wrote, or one in an earlier search result or page. The URL must match exactly, so the model cannot guess or alter an address. It reads text/html, text/plain, text/markdown and PDF pages, up to 5 MB, within 15 seconds, and returns the text as Markdown with its links, cut at 48,000 characters. It follows up to 5 redirects, and a page that disallows automated access in its robots.txt is refused. An agent can read at most 20 pages in one run. Each fetch counts once on the usage rate card.

Pages are fetched from their own servers, which may be outside the EU. Page content is untrusted: the tool tells the model to treat it as information, never as instructions.

When a read fails, the tool answers with one of these error codes so the model can recover:

The console's tool checklist also offers backbone.extraction (config.schemaIds) and backbone.document_convert. Both envelopes are accepted and stored on the agent version, but no executor ships for them yet, so they are not offered to the model during a run. They become active without any change on your side once their executors land.

The sandboxed shell

backbone.bash gives the model a Bash tool that runs commands in a sandbox of its own conversation: a disposable container with no network access, offered only where the deployment enables it. The envelope takes one setting for who confirms a command, approval:

A version saved without approval is stored with sandbox. A version stored before the setting existed has none and reads as pause, so no deployed agent starts running commands unattended.

Delivered files

An agent with the sandboxed shell (backbone.bash, offered only where the deployment enables it) collects every regular file its commands write under outputs/ as an agent_output file. The function_call_output item of that call lists them in output_files, each with file_id, name, content_type, bytes, kind, delivery and preview. A symlink or another special file is named in the result and not collected, and a call collects at most 100 files and 256 MiB (2 GiB per conversation).

The shell envelope takes one setting for those files, outputs:

With outputs: held the server injects a Deliver tool into the conversation. You do not configure it, and a backbone.deliver entry in tools is dropped. The model calls Deliver with 1 to 20 file_ids of held outputs of this conversation and a short note. A call that names anything else fails in the same turn with the reason, and raises no approval.

Deliver is human-only. Every shortcut is off for it: no conversation mode, automatic approver or "allow for this chat" grant approves it. A deny rule, classes.write: block and plan mode still block it. It pauses for a person, and the pending backbone:approval_request item carries human_only: true and a deliver card, { files: [{ file_id, name, content_type, bytes, preview }], note }.

Only the person who asked for the files decides it, approve or reject: their own signed-in session, or the surface caller of the same scope. Another member, an admin and an API key, the requester's own included, get 403 deliver_requester_only and the approval stays pending. Approving flips the files to delivered; rejecting leaves them held and tells the model why.

The hold is per conversation and only ever tightens. Once a held turn has run, the conversation stays held: switching the agent to immediate afterwards does not release files already written, and a copy of a held file is held too.

Viewing images

Wherever the sandboxed shell is offered, the server also injects a ViewImage tool, so the model can look at an image its commands produced: a chart it plotted, a rendered page, a screenshot. You do not configure it, and a backbone.view_image entry in tools is dropped.

The model calls ViewImage with one path, relative to the sandbox's working directory, such as outputs/chart.png. An absolute path, a path with .. and anything under .backbone/ are refused. The file must be a PNG, JPEG, WebP or GIF of at most 20 MB, and a command must have run first; a call before any command answers no sandbox yet: run a command first. A call that succeeds answers one line:

image `outputs/chart.png` attached as `file_607fe7c92e2b4d599d8f7b1a789dddd8` (1200×800)

The image is stored as an agent_output file of the conversation, under the same hold as the conversation's other outputs, and the model sees it as a picture from its next step on, next to a line with its path and size. Right after the call's function_call_output, the response carries a backbone:tool_image item with six fields, which a client uses to show the image beside the step:

{
  "type": "backbone:tool_image",
  "id": "timg_call_5",
  "call_id": "call_5",
  "file_id": "file_607fe7c92e2b4d599d8f7b1a789dddd8",
  "path": "outputs/chart.png",
  "width": 1200,
  "height": 800
}

The item is output only. The server builds it from its own records, and a request that sends one as input is refused with 400 server_produced_item.

A turn attaches at most five images; the sixth ViewImage call fails with the reason and the model works from what it has. Images a connector returns share the ten-per-call limit below but not this budget of five. Viewed images share the per-call budget with attached images: every model call carries the ten most recent images of the conversation, viewed and attached together, and an older one keeps its line only. An image whose file has expired or can no longer be read reaches the model as a line saying it is no longer available. If the model does not accept image input, the turn is retried once without pictures, as with attachments: the model then reads only the lines.

Showing tables and charts

Add { "type": "backbone.show_table" } or { "type": "backbone.show_chart" }, or both, to tools and the model gets a ShowTable or ShowChart tool. It passes the data; the person sees a table, or a bar or line chart, in the conversation, in the chat app and in the embedded panel, and sees it again when the conversation is reopened. Neither envelope takes options, and an agent version that does not list one never offers it. Both tools are read, so they run without a pause under the default policy; a policy rule can tighten them like any other tool.

ShowTable takes:

A number, currency or percent column takes numbers, and percent values are fractions (0.25 is 25 %). A date column takes YYYY-MM-DD dates, shown as that calendar day in every time zone. A text column takes strings or numbers. Numbers are shown in the viewer's locale.

ShowChart takes:

Every number is finite and at most 1e15 in absolute value, and the arguments of one call are at most 64 KiB. A call that breaks a rule fails with one reason the model can act on, such as rows: 214, the limit is 200, and nothing is shown. A call that succeeds answers with a receipt, not the data, such as Shown to the user: table "Open invoices", 2 rows.

Every string is shown as plain text: no Markdown, HTML, links, images or colours. A long table shows its first 20 rows and the rest on request, a chart's values can be read as a table (Show data), and a table can be copied as CSV (Copy as CSV), with every text cell a spreadsheet would read as a formula prefixed with '.

Right after the call's function_call_output, the response carries a backbone:ui item with the checked data in props:

{
  "type": "backbone:ui",
  "id": "ui_call_7",
  "call_id": "call_7",
  "component": "table",
  "schema_version": 1,
  "props": {
    "title": "Open invoices",
    "columns": [
      { "key": "number", "label": "Invoice", "format": "text", "align": "left" },
      { "key": "amount", "label": "Amount", "format": "currency", "currency": "EUR", "align": "right" }
    ],
    "rows": [
      { "number": "INV-1042", "amount": 1200.5 },
      { "number": "INV-1043", "amount": 80 }
    ]
  }
}

Render from the item's props, never from the call's arguments: only props has been checked against the rules above. component is table or chart, and props has the shape of that tool's parameters. A client that meets another component, or a schema_version above the one it knows, should show a short line in its place, not the raw JSON. The item is output only: a request that sends one as input is refused with 400 server_produced_item, and it is never sent to the model. The CLI prints each table, and each chart as its data table, under its title; the run envelope of the CLI and the MCP server, and n8n's simplified output, list the items under components, each with id, callId, component, schemaVersion and props.

Memory

Add a memory envelope to an agent version and the agent remembers things about the organization member it talks to — preferences, role, recurring context — across conversations and across agents. The model uses a memory tool with the same commands as Anthropic's memory tool (view, create, str_replace, insert, delete, rename) under /memories.

{ "type": "backbone.memory", "access": "read_write" }

There is one memory per member per organization, shared by every agent that has the envelope. Give an agent that reads untrusted input (web search, connectors, uploaded files) read, so it cannot write into what your other agents read.

Who has memory. A signed-in member in the console always does. An API-key request only when it sends the header X-Backbone-Memory: enabled, exactly once — a backend that serves many people on one member's key would otherwise fill that member's memory with other people's facts. Surface end users never have memory. A continuation or an approval decision decides again from its own request.

What is recorded. The model reads the real memory during the run. Everything stored afterwards holds placeholders: the response, the conversation items, tool spans and approval outcomes show {"command":"view"} and [memory view: ok], never a path or a file's content. A member reads their own memory through the memory API.

Retention. A memory file that nobody has read or written for 180 days is deleted automatically, by a nightly job. A read is an agent opening the file with view, or the member opening it in My memory, with 2kw memory cat or through the memory API; every write counts too. Listing a folder does not count: an agent's view of a directory, the folder tree in My memory and 2kw memory ls leave the file's age unchanged.

Limits of the redaction.

• The agent's own replies may repeat what it remembered, and they are ordinary conversation content.
• When your organization captures prompts in tracing, model-call spans contain the model's full input, memory tool results included.
• A memory call that your approval policy pauses shows its arguments in the approval request, because the approver has to see what they approve. Memory calls do not pause unless your hitl_policy asks for it.
• An approved memory call runs only when the member who raised it approves it, from a request that may use memory. An approval by someone else, or by an API key without X-Backbone-Memory: enabled, is refused with HTTP 409 memory_release_refused, and the approval stays pending, so the member can still approve it. Anyone who may decide the call can still reject it.

Function tools (your webhooks)

A function tool is executed by calling an HTTPS endpoint you operate. The model sees a normal function definition; 2kw.ai calls your endpoint with the arguments and returns your reply to the model.

{
  "type": "function",
  "name": "lookup_order",
  "description": "Look up an order by its number and return status, items and delivery date.",
  "parameters": {
    "type": "object",
    "properties": { "order_number": { "type": "string" } },
    "required": ["order_number"]
  },
  "endpoint": "https://erp.example.com/hooks/lookup-order",
  "secret": "a-long-random-string",
  "annotations": { "readOnlyHint": true, "destructiveHint": false, "openWorldHint": false }
}

Your endpoint receives:

POST https://erp.example.com/hooks/lookup-order
Content-Type: application/json
X-Backbone-Signature: sha256=9f2c…
Idempotency-Key: resp_…:call_…

{
  "tool": "lookup_order",
  "call_id": "call_…",
  "organization_id": "org_…",
  "arguments": { "order_number": "SO-2026-0417" }
}

Whatever your endpoint returns with a 2xx status becomes the tool result the model reads, as a string. Timeouts, connection failures and 5xx answers are retried once, then reported to the model as a failed tool so it can tell the user your system was unreachable. A 4xx is taken as a deliberate rejection and is not retried. Redirects are not followed: a 3xx is reported to the model as a failed tool. The per-attempt timeout is 20 seconds.

Idempotency-Key is what identifies one call across attempts: {response_id}:{call_id} normally, and approval:{approval_id} for a call executed after a human approved it. Every attempt of the same call repeats the same key — the retry after a 5xx, and a run that continues after a restart. Delivery is at-least-once, so a receiver that creates records or charges money should store the key and return the first result when it sees it again. call_id alone does not identify a call: it is unique within one response, not across them.

arguments are the model's arguments parsed as JSON. If the model emits arguments that do not parse, you receive the raw string and decide what to do with it.

Client-executed tools

On a direct provider/model call you can pass tools on the request the way you would with OpenAI. 2kw.ai does not execute those: the response comes back with status: "requires_action" and a function_call item per call. Run the tool yourself and continue:

{
  "model": "openai/gpt-4o",
  "previous_response_id": "resp_…",
  "input": [
    { "type": "function_call_output", "call_id": "call_…", "output": "{\"status\":\"shipped\"}" }
  ]
}

Set "failed": true on a function_call_output to tell the model the call did not succeed. Every pending call must be answered on the continuation, or the request is refused with incomplete_tool_outputs.

The embeddable surface uses the same mechanism to run tools inside the host application under the end user's own session. The approval policy gates these tools exactly like server-side ones: a write-class call is withheld until someone approves it, and the approved function_call is handed to the host on the continuation that carries the decision; a destructive one is refused.

The tool loop

One turn may take up to 25 rounds of tool calls. After the last one the model answers from what it has, without calling more tools, and the response ends incomplete with incomplete_details.reason max_tool_iterations. Calls in the same round run concurrently. At most 128 tools are offered to the model on a run.

Tool discovery

An agent embedded in a host application can have hundreds of tools available through the host's tool catalog. Offering all of them on every call would cost tokens and confuse the model, so a run starts with a small set and finds the rest when it needs them:

• The model starts with the agent's own tools and the catalog's core tools. It finds the others with the built-in search_tools tool, by a plain-language query, and can then call what it found. Tools found earlier in a conversation stay callable on later turns.
• However the model reaches a catalog tool, your host receives the call as a function_call under the tool's own name, with the tool's own arguments — exactly as for any relay tool.
• Function tools declared on the agent itself are always available; discovery applies to catalog tools.
• search_tools and call_tool are reserved names. A catalog cannot use them, and an agent version cannot add a tool named call_tool.

Page context

A client that knows what the user is looking at can say so with a backbone:ctx input item:

{ "type": "backbone:ctx", "ctx_type": "invoice", "ctx_id": "9912" }

The model is told which page the user is on, with your values marked as data rather than instructions. Send the item on every turn: a page that has not changed adds nothing, and each change is added once, where it happened in the conversation, with the newest one in effect.

When the user moves to a page that shows no single object — a list, a dashboard — say so with an empty item, which clears the context:

{ "type": "backbone:ctx" }

Sending no item at all is a different statement: it means the page has not changed, which is what keeps the context alive across the tool calls of one turn.

Invoking a skill

A client can say that the user picked one of the agent's skills for this turn with a backbone:skill input item, placed before the user's message:

{ "type": "backbone:skill", "name": "review" }

The model is told to load that skill before answering. The item applies to the turn it arrives in, including the tool calls that turn makes; the next user message without it is an ordinary turn again. The invocation stays in the conversation's history as a record of that turn and is not repeated.

The name must be one of the skills the agent binds. An unknown name, a missing name or a second backbone:skill item in one request is refused with 422 and the code invalid_skill_invocation, and the message lists the skills the agent does bind. Invoking a skill never changes which skills the run carries.

File Search and Citations

With file_search configured, the model can search the agent's knowledge bases. Each result carries a handle such as h_0, and the model is instructed to cite the handles it used inline as [h_0].

The tool result the model reads:

{
  "results": [
    {
      "id": "h_0",
      "text": "Mounting. Tighten the four M8 bolts to 25 Nm in a cross pattern …",
      "document": "service-manual-2026.pdf",
      "pages": [14, 14],
      "headings": ["Installation", "Mounting"]
    }
  ],
  "truncated": false
}

text is the passage with its surrounding context. truncated: true tells the model that lower-ranked hits were dropped to stay within the result budget, so it does not treat a partial list as the whole corpus. When retrieval had to fall back to keyword search, a degraded object says so.

Citations

For every handle the answer actually cites, the response carries a backbone:citation output item next to the message, with the knowledge base, document, revision, chunk, document name and page range. The [h_0] marker stays in the text so a client that only renders text still shows where a claim came from; a client that understands the item renders a footnote.

Resolve a citation to its passage at any time with GET /v1/knowledge-bases/{knowledgeBaseId}/citations/{chunkId}. The passage stays resolvable after the document is re-ingested, and after it is deleted the citation still resolves with its text withheld. Details are on the Knowledge page.

Handles are numbered per run, so two searches in one turn never reuse a handle, and a handle the model invents resolves to nothing.

Files in a Turn

Upload a file once with POST /v1/files, then attach it to a turn with an input_file part that names its id:

{ "type": "input_file", "file_id": "file_607fe7c92e2b4d599d8f7b1a789dddd8" }

The model sees one line per attachment with filename, type, size and id; images additionally reach it as pictures. Upload, retention, limits, how a file reaches a tool and the errors are on the Files page.