• Gemini API Tool Retrieval: Agents With 200 Tools Use Almost 7x Fewer Tokens in Google’s Example

    Tool Retrieval Gemini Api

    Google announced Tool Retrieval in the Gemini API – a mechanism that lets an agent keep its entire tool library on standby, while the full parameter descriptions enter the model’s context only when a task actually needs them. The feature is aimed at AI agents with dozens or hundreds of tools and connected MCP servers, and is available in preview in the Interactions API.

    In a worked example from the documentation – an illustration, not a measurement – a conversation with an agent that has a library of 200 tools costs about 23,650 input tokens instead of 164,000, almost 7x fewer. Anthropic claimed an 85% reduction in token usage for its counterpart, and OpenAI reported 47% at the same accuracy on the MCP Atlas benchmark, but the companies measured under different conditions. The savings come down to what exactly the model sees before it searches.

    How Tool Retrieval works

    A developer adds a tool_search tool to the request and sets defer_loading: true on functions that are rarely needed. The model still sees the name and short description of every deferred function – about 26 tokens per tool in Google’s early testing – but not its parameter schema. When a function is needed, the model requests it by name, the API loads the schema, and the model makes the call, all within a single request.

    The same flag can be set on an entire remote MCP server, in which case all of its tools are loaded on demand. Loaded schemas are appended as the conversation goes on rather than placed at the start of the prompt, so the prompt prefix stays the same and keeps being served from the cache.

    For companies with their own tool registry or per-user permission filter, there is an application-side search mode, execution: "client". Each search in this mode adds an extra round trip, but tools don’t have to be declared up front: they can be added mid-conversation, for example when a user connects a new integration.

    How Tool Retrieval differs from the Anthropic and OpenAI equivalents

    Anthropic shipped its Tool Search Tool almost a year before Google, and OpenAI added tool search to the Responses API alongside GPT-5.4. Gemini’s interface closely mirrors OpenAI’s – the same tool_search type, defer_loading flag and execution parameter – which, in our editorial assessment, should make it easier to move agents between platforms.

    Parameter Gemini API, Tool Retrieval Claude API, Tool Search Tool OpenAI Responses API, tool search
    What the model sees before searching Name and description of every deferred function Only the search tool and non-deferred tools Function names and descriptions; for namespaces and MCP servers, only an overall description
    How it finds a tool Requests schemas by exact name Regex or BM25 search, up to 5 results by default Loads the needed functions, namespaces or MCP servers
    Models Gemini 3.6 and later, including Gemini 4 Neon Claude 4.5 and later GPT-5.4 and later
    Claimed token savings, % Not claimed; about 86 in the worked example Over 85 47

    The Gemini model sees the whole catalog from the start, but that catalog is paid for on every request: 1,000 tools at 26 tokens each come to about 26,000 tokens (editorial estimate). With Anthropic, deferred tools stay out of the context entirely, and a single request can include up to 10,000 of them. Google provides no measurements of tool-selection accuracy, even though it names declining accuracy as one of the reasons for the launch; Anthropic reported that in internal testing Claude Opus 4.5’s accuracy on MCP tasks rose from 79.5% to 88.1%.

    When deferred loading pays off – and when it doesn’t

    There is no separate charge: tokens are billed at the model’s standard rates. But each server-side search is another model pass over the full context, which in effect means paying for the prompt almost a second time, minus the cached portion.

    An editorial calculation based on Google’s example: Gemini 4 Hydrogen, the model used in the announcement, costs $1.50 per 1M input tokens. The 200-tool conversation comes to $0.246 with standard loading and about $0.035 with Tool Retrieval – roughly $2,100 saved per 10,000 conversations. In Google’s counter-example – 10 tools and a 30,000-token context – the total is 92,830 tokens instead of 68,000, or 36.5% more. Output tokens and caching are not included; Hydrogen’s cached input costs $0.075 per 1M tokens, 20 times less.

    Preview limitations: Tool Retrieval doesn’t work with the classic generateContent method, and the interface may change before general availability. A loaded schema cannot be unloaded until the conversation ends. Google recommends keeping standard loading for libraries of up to roughly 20 stable tools, while Anthropic advises turning on search from as few as 10.


    bnr2mob
    Subscribe
    Notify of
    guest

    0 Comments
    Oldest
    Newest Most Voted