Cost
Reduce the token cost of web-searching agents
Teams optimise the wrong number here. The API bill is legible and generally modest; the token bill attributable to retrieval is larger, less visible, and easier to reduce.
Verified 5 August 2026
One: return only the fields you read
A full hit carries around twenty fields. If your agent reads url, title and snippet, the other seventeen are context you paid for and the model has to ignore.
Keep independent_sources if the agent makes trust judgements — it is small and carries more decision value per token than the snippet. Keep id if you will fetch documents.
curl -H "x-api-key: $UNLOB_API_KEY" \
"https://api.unlob.com/search?q=rust+async&fields[]=id&fields[]=url&fields[]=title&fields[]=snippet&fields[]=independent_sources"Two: fetch document text lazily
The single largest saving available. A search that returns page bodies spends the context window before the agent has chosen; a search that returns snippets lets it choose first.
This is why search here is metadata-only and /doc/:id is separate. Ten snippets and one document fetch is a fraction of ten documents.
Three: filter harder than feels comfortable
Under-filtering feels safe because a narrower query might miss something. It might — and the alternative is paying for noise on every call, in context and in the reasoning spent rejecting it.
A quality floor, a content type and a corroboration threshold typically remove more junk than any amount of query tuning. Filters are also free: they narrow the candidate set before scoring, so a filtered query is often faster than an unfiltered one.
curl -H "x-api-key: $UNLOB_API_KEY" \
"https://api.unlob.com/search?q=oauth+refresh+token&content_type[]=docs&min_quality=70&limit=5"Four: collapse duplicates
On news, finance or press-release-driven topics, syndication can fill an entire result set with one item. collapse=story converts ten wasted slots into ten distinct facts, which is the cheapest quality improvement available on those queries.
Five: collapse the retrieval loop
A multi-step agent that searches, reads, searches again and reconstructs relationships is spending most of its tokens on reasoning the index could have answered directly.
assemble_context runs that loop server-side and returns a packed set with reasons attached. Where it fits, it replaces a dozen calls and all the intermediate context they generated.
curl -H "x-api-key: $UNLOB_API_KEY" \
"https://api.unlob.com/assemble_context?q=your+question&budget=3000"Measure it
Log tokens attributable to retrieval separately from tokens spent on the task. Most teams find retrieval is a larger share than expected, and that the largest single line is document text fetched before selection.
Use limit=0 for coverage probes — it returns the total and facets with no results at all, which is enough to decide whether a query is worth running properly.
Frequently asked questions
What is the single biggest saving?
Fetching document text lazily. Returning ten page bodies when the agent needed one is the most expensive mistake in agent retrieval, and it is a one-line change.
Does aggressive filtering hurt recall?
It reduces raw recall and usually improves output quality, because the results removed are ones the agent would have read and rejected. Measure against your own queries rather than assuming either way.
How many tokens does a search result cost?
A projected metadata hit is roughly 50 to 100 tokens; a full page body is frequently 2,000 to 8,000. That ratio is the entire argument for two-step retrieval.
Try it against your own queries
10,000 free requests a month, no card. Everything in this guide works on the free tier.