Choosing a web search API

Five categories, not one market

Own-index APIs crawl and index the web themselves — Exa, Parallel, Brave, Mojeek, unlob. Nobody upstream can deprecate them.

Platform grounding bundles search into a model platform — Gemini, Perplexity, Azure. Nothing to integrate if you are already there, and no control if you are not.

Retrieval layers sit on top of other providers and add cleaning and citation — Tavily, Linkup. Convenient, with someone else's index underneath.

SERP resellers return Google or Bing results — SerpAPI, Serper, DataForSEO. Right when you specifically need Google's ranking.

Extraction tools turn a URL into clean text — Firecrawl, Jina Reader. Essential in most stacks, and they need something to hand them URLs.

Question one: who owns the index

A provider without its own index inherits three risks it cannot control: the upstream can be deprecated, repriced, or rate-limited. The Bing retirement is what the first looks like when it lands — eight sub-APIs gone, and every product built on them rewritten regardless of how good the results were.

This does not make resellers wrong. It makes them a dependency you should choose deliberately rather than by default.

Question two: what shape is the response

A SERP reseller returns links, so you need extraction. A retrieval layer returns cleaned chunks. A grounding API returns an answer. An agent-first index returns passages with metadata, and lets you fetch text for the ones you chose.

Match this to what your agent actually consumes. Paying for a synthesised answer and then parsing it back into sources is a common and expensive mistake.

Question three: what trust signals come with it

If the API gives you nothing but relevance, your application has to judge source quality itself — usually by prompting the model, which is the least reliable option available.

Ask specifically: is there a publisher authority score, a per-page quality score, a corroboration count, and a way to find out why something is absent. Each one you do not get is something you will build.

Question four: cost at your real volume

Model your actual call mix rather than the headline rate — including tokens, extraction and the retries caused by poor first results. And check the free tier: it determines how much you can evaluate before committing, and it is the thing most likely to change.

Question five: what happens when the vendor changes

In eighteen months this category saw an API retired, a free tier removed and a vendor acquired. Assume at least one of those happens to whoever you pick.

The practical defences are boring and effective: keep retrieval behind an interface, do not scatter provider-specific parameters through your codebase, and prefer providers whose continuity does not depend on someone else's decision.