Claude Haiku 4.5's API Search Ran on 99% of Top-List Questions but 22% of Problem Descriptions

When Claude is given a web search tool and a buyer's question, it does not always search. Across 5,054 runs of 1,008 buying questions in the Machine Relations Index, Claude Haiku 4.5 ran at least one search in 98.8 percent of top-list questions ("top workflow automation platforms for...") and in 22.2 percent of problem-first questions ("every department in our company runs on a different tool and nothing talks to each other"). When it did search, it did not search the buyer's sentence. Its queries averaged 7.5 words, against 17.0 words in the sentence they came from, and 23.8 percent of them carried a year, almost always one the buyer had not written. The same questions sent to the ChatGPT API route, where our request requires a search, produced one search per run and that search was the buyer's sentence word for word, in all 5,119 runs.

These are two API routes the Index uses, one model each. They are not the Claude or ChatGPT apps, and the page says where that limit matters.

What was measured #

The Index asks the same library of buying questions of each engine on a recurring schedule and records the answer, the sources it cites, and, where the engine reports them, the search queries it ran. Search queries were recorded for the Claude and ChatGPT routes from September 1, 2026, so that is where this window starts.

• Window and size. September 1 to October 31, 2026. 1,008 questions in six question shapes (168 each) across 24 categories. Claude: 5,054 successful runs. ChatGPT: 5,119.
• Claude route. Claude Haiku 4.5 (claude-haiku-4-5-20251001) through Anthropic's API with the web_search_20250305 tool available and the choice of whether to use it left to the model. Anthropic's documentation says Claude determines when to search based on the prompt (Anthropic, web search tool).
• ChatGPT route. gpt-4.1-mini-2025-04-14 through OpenAI's Responses API with the web_search tool and tool_choice set to required in the request as currently configured; all 5,119 runs searched, consistent with that setting across the window. OpenAI describes this kind of non-reasoning web search as the model sending the user's query to the search tool (OpenAI, web search). Because the search is required, the ChatGPT route cannot show whether ChatGPT would have searched.
• Searched. A run counts as searched when the ledger recorded one or more search queries for it. An unsearched Claude run answered from the model alone and cited nothing: all 1,113 of them have zero recorded citations.
• Not included. Gemini, Perplexity and the two Google surfaces. Gemini's API lane recorded searches for 516 of 4,501 runs in this window, and Perplexity recorded none of 5,136, so there is nothing comparable to count.

Result 1: Claude's decision to search depends on the shape of the question #

The pattern follows how much the sentence already names a thing to look up. A top list, a "best" question and a head-to-head comparison nearly always send Claude to the web. A problem description that names no product or category sends it there about one time in five, and the 665 problem-first runs that did not search produced an answer with no citations at all.

It is mostly a property of the question, not of the day. Among questions with at least five runs, Claude never searched 80 of 130 problem-first questions and searched every time on 11. For top lists it searched every time on 120 of 124.

The middle shapes are the unstable ones. For "how to choose", 53 of 124 questions were searched on some runs and not others, so the same question can produce a cited answer on one run and an uncited one on the next.

Result 2: Claude searches a short keyword query, not the buyer's sentence #

The 3,941 searched Claude runs ran 6,861 search queries: one query in 1,288 runs, two in 2,403, three in 236 and four to six in 14. A query averaged 7.5 words. The buyer sentences they came from averaged 17.0. Three of the 6,861 queries were the buyer's sentence unchanged.

Claude added words the buyer did not write. These are the most frequent, as a share of all 6,861 queries, counting a query when it contained the word and its question did not:

It also added a year. 1,632 queries (23.8 percent) contain a year from 2020 to 2029, and in 25 of them the buyer's sentence had already named one. The year distribution is the least expected part:

Queries can name more than one year. 552 queries name 2024 or 2025 and not 2026, which is 33.8 percent of the year-stamped queries, in a window that ended in late 2026. The year shows up most often on "best" questions (41.9 percent of their queries) and top lists (33.5 percent), and least on "how to choose" (7.6 percent). The ledger shows what Claude searched. It does not show which pages those searches returned, so it cannot say what a stale year does to the results.

Result 3: the ChatGPT API route searched the question as written #

In all 5,119 ChatGPT runs the model ran exactly one search, and the search text equals the buyer's question, ignoring case and surrounding spaces. There is no shortening, no added word and no added year. That is a property of this route with search required, not evidence about the ChatGPT app, whose search behavior this page did not measure. The figure says only that when a non-reasoning model is required to search, the wording of the question is the wording of the query.

What this changes for someone tracking AI citations #

Three readings, each limited to these routes.

1. A zero on a problem-first question is mostly a no-search answer. On the Claude route 77.8 percent of problem-first runs never reached the web, so a tracked brand missing from those answers did not lose a citation to a competitor. The answer contained none. A panel made mostly of problem descriptions will read low on this route for every brand.
2. The sentence you track is not the query that runs. On the Claude route the search is a seven-word keyword query that often leans on "comparison", "best", "cost" and a year. On the ChatGPT route the search is the sentence itself. Tracking one prompt wording is tracking one input to a step that, for Claude, rewrites it. The Index's study of rewording found a reworded question shares 39.4 percent of cited-source volume where a plain rerun shares 46.1 percent.
3. Whether the question triggers a search is a separate measurement from whether a brand is cited. Reporting the share of runs that searched beside the share that cited a brand separates "the engine did not look" from "the engine looked and chose someone else".

What this page does not show #

• Not the apps. Claude Haiku 4.5 through the API with a search tool is not Claude.ai, and a required search on gpt-4.1-mini is not ChatGPT. Other models, other tool settings and the consumer apps can search at different rates and write different queries.
• Not which pages the searches returned or why one was cited. The ledger records the query and the cited domains, not the ranking behind them.
• Not buyer demand. The 1,008 questions are the Index's library, written to a fixed set of shapes. They are not a sample of what buyers ask.
• One model per engine and two months. Haiku 4.5 is Anthropic's small model. Larger Claude models may decide to search differently.
• The ledger is not public. The public Index file carries no per-answer rows. These figures were computed from the Index answer ledger, export ledger_export_v1 of 2026-10-31T22:46Z (pub_query_runs partitions 2026-09 sha256 060af35f… and 2026-10 sha256 51bbdfd0…), through the query below. The year counts are plain substring matches on 2024, 2025 and 2026.

How the main table was computed #

select query_family,
       count(*) as runs,
       count(distinct query_library_id) as questions,
       sum((search_query_count > 0)::int) as searched,
       round(100.0 * avg((search_query_count > 0)::int), 1) as pct_searched,
       round(avg(citation_count), 2) as mean_citations
from pub_query_runs
where engine = 'claude'
  and provider_model = 'claude-haiku-4-5-20251001'
  and success
  and search_queries is not null
  and run_date between '2026-09-01' and '2026-10-31'
group by 1;

search_queries is a JSON list of the queries the engine ran. Query length, added words and years come from unnesting that list against query_text. Question shapes are the ledger's query_family.

Frequently asked questions #

Does Claude search the web for every question? #

Not in the Index's API route. With a search tool available, Claude Haiku 4.5 searched 78.0 percent of 5,054 buying-question runs: 98.8 percent of top-list runs and 22.2 percent of problem-first runs. The 1,113 runs that did not search cited no sources.

Does Claude search the exact words a buyer typed? #

Almost never on this route. Three queries of 6,861 matched the question. Queries averaged 7.5 words against 17.0 in the question, often added words such as "comparison" or "best", and carried a year in 23.8 percent of cases.

Does ChatGPT rewrite my question before it searches? #

On the ChatGPT API route the Index uses, no: with search required, all 5,119 runs ran one search that equalled the question. That says nothing about how the ChatGPT app behaves.

Why would a brand not be cited on a question it should win? #

One reason is that the engine did not search. On the Claude route, 665 of 855 problem-first runs produced an answer without a search and so without a citation. Report the share of runs that searched next to the share that cited the brand before reading a missing citation as a loss.

Where do the numbers come from? #

From the Machine Relations Index answer ledger, September 1 to October 31, 2026, as described above. Quote the window, the route and the model with any figure.

The short version #

• Claude Haiku 4.5 with a search tool searched 98.8 percent of top-list questions and 22.2 percent of problem descriptions.
• Its search queries ran 7.5 words against 17.0 in the question, with added words like comparison and best and a year in 23.8 percent.
• The ChatGPT API route, forced to search, searched the question verbatim in all 5,119 runs.
• A missing citation on a problem-first question is mostly a run that never searched.
• These are API routes with one small model each, not the apps.

Machine Relations is the discipline of earning AI citations and recommendations for a brand by making that brand legible, retrievable, and credible inside AI-driven discovery. Jaxon Parrott coined it in 2024; AuthorityTech, which he founded in 2018, is The World's First Machine Relations Agency; Machine Relations is open to anyone and machinerelations.ai is its home.