Search the site
All 224 pages on unlob.com, grouped below. Press / anywhere on the site to search them.
Search pages, guides, comparisons…/
Looking for an endpoint, a parameter or an MCP tool? That is in the API and MCP reference ↗.
Product and company 18
- The evidence layer for agents.The evidence layer for agents: ground an objective and get provenance-aware, deduplicated, token-budgeted evidence — and a receipt saying when it is not enough.
- GroundingOne call: provenance-aware, deduplicated, token-budgeted evidence with a coverage receipt.
- PricingPlan table, per-1,000 rates and how they compare.
- Coverage graphThe evidence graph over the open web: origins, owners, and six traversal operations.
- Coverage transparencyAsk why any URL is or is not in the index and get a real answer.
- Web search APIs for AI agents — the 2026 landscapeSourced, vendor-neutral map of the market: index ownership, price, capability and funding.
- Why an own index mattersDeprecation, repricing and rate-limit risk — and why owning an index is no longer sufficient on its own.
- Cost modelThe unit economics behind $0.40 per 1,000 queries.
- BenchmarksMeasured latency, throughput and memory, with the conditions attached.
- How it worksAdmission control rather than ranking: how the index is built and served.
- Sovereign evidence infrastructureThe evidence layer deployed inside your own boundary: your index, your source policy, your query record. Bring your own model.
- UnlobBotHow we crawl, and how to allow, restrict or block it.
- FAQAnswers about the index, pricing, the coverage graph and compliance.
- AboutWhat unlob is, what it is not, and how we operate.
- ContactSupport, removal requests, crawler questions and corrections.
- ChangelogWhat changed in the API and the index, most recent first.
- Terms of serviceThe service, accounts and API keys, acceptable use, and fees and billing.
- Privacy policyWhat we collect, what we do not do, how long we keep it, and your rights.
Use cases 12
- Use casesWorkloads the API suits, with the parameters that make each one work and where it is not the right tool.
- RAG over the open webRetrieve, deduplicate, trust-rank and pack to a token budget — either as separate calls you control, or as one call that does the whole loop.
- Autonomous research agentsGraph traversal replaces the search-read-search loop: triage by centrality, build entity briefs in one call, and follow edges rather than guessing queries.
- Fact-checking with corroboration countsCount distinct hosts asserting a claim rather than counting copies — and get the count from an index where deduplication does not destroy it.
- Monitoring topics and entitiesBrowse by recency with no query at all, bound by publication date, and collapse syndication so a monitoring feed carries distinct events.
- Competitive and market intelligenceEntity dossiers, co-mention discovery and connecting paths — the operations that make competitive research structural rather than a search loop.
- Documentation search for coding agentsThe code vertical and the docs content type remove tutorial blogspam; the term filter makes error-code lookup exact.
- Grounded question answeringGet a packed, corroborated context set in one call, with a reason attached to every passage so the answer can be explained.
- Regulatory and compliance researchTime-bounded queries, authority filters, uniform corroboration thresholds and an endpoint that explains any exclusion.
- Finding sources worth followingCentrality surfaces what a field treats as foundational; similar and related widen from a seed without inventing new queries.
- Serving users in many languagesOne embedding space for 101 languages means retrieval crosses languages without a translation step or a per-language index.
- Grounding sovereign and public-sector agentsThe evidence layer deployed inside your jurisdiction: no query leaves, the record of what was asked is yours, and the model reasoning over the evidence is your own.
Guides 15
- GuidesAgent retrieval, cost, trust and MCP, written from engineering practice.
- Web search for AI agentsAgent retrieval inverts the assumptions of consumer search: precision matters more than recall, metadata beats full text, and absence has to be explainable.
- GraphRAG without building the graph yourselfGraphRAG normally means extracting entities from your corpus and running a graph database; a search API shipping a graph over the open web removes that step.
- Stop agents hallucinating from search resultsRetrieval reduces hallucination without eliminating it. Two structural fixes, corroboration counting and explainable absence, reach what prompting cannot.
- Reduce the token cost of web-searching agentsMost of what an agent spends on retrieval is not the search call — it is the context consumed by results it did not need and the reasoning spent rejecting them.
- Search API pricing explainedCredits, per-request fees, per-token charges and processor tiers make headline rates non-comparable. Here is how to normalise them.
- Choosing a web search APIFive questions that decide the choice: index ownership, response shape, trust signals, cost at your volume, and what happens when the vendor changes.
- Auditable retrieval for regulated workIn regulated work the question after an answer is what it missed. Four capabilities make that answerable, and ranking quality is not one of them.
- Build a research agentA research agent that searches, reads, and searches again spends most of its budget rebuilding structure the index already has. Graph traversal collapses that loop.
- Web search over MCPOne config block gives any MCP client the evidence layer — ground and get_document on the grounding profile — or every tool, including six graph operations no other search API exposes.
- Multilingual search for agentsOne embedding space for 101 languages means a query in one language retrieves relevant passages in another — no translation, no per-language index.
- Measuring an index before you depend on itCoverage claims are unverifiable by construction. Here is a measurement you can run yourself in an afternoon, on any provider, using your own URLs.
- Retrieval that survives a provider shutting downThree providers changed terms or disappeared in eighteen months. Teams that moved cheaply had drawn one boundary in advance, and it was not an HTTP wrapper.
- Getting recency right in agent retrievalFour different dates hide behind the word recent, and filtering on the wrong one is how a correct retrieval stack returns confidently outdated answers.
- Grounding a sovereign model without sending the question abroadA model kept inside a jurisdiction that grounds itself through a search API outside it has moved the question, not the data. What a search call discloses, what a provider can and cannot promise, and how to keep the record inside the boundary.
Comparisons 25
- Comparisons
- unlob vs Parallel Search
- unlob vs Exa
- unlob vs the Brave Search API
- unlob vs Tavily
- unlob vs Grounding with Google Search
- unlob vs Perplexity Sonar
- unlob vs Serper
- unlob vs SerpAPI
- unlob vs Firecrawl
- unlob vs Linkup
- unlob vs Valyu
- unlob vs Diffbot
- unlob vs Bright Data
- unlob vs DataForSEO
- unlob vs Google Custom Search JSON API
- unlob vs Grounding with Bing Search
- unlob vs the OpenAI web search tool
- unlob vs Jina Reader
- unlob vs Mojeek
- Exa vs Parallel Search
- Exa vs Tavily
- Brave Search API vs Exa
- Tavily vs Linkup
- Serper vs SerpAPI
Alternatives 21
- Web search API alternativesThe alternatives to every web search API, with dated prices and index ownership.
- Exa alternativesNeural, embeddings-first search over a proprietary index, sold alongside content extraction, answers and agentic research products.
- Parallel Search alternativesA suite of web search, extraction and research APIs on a proprietary index, from the former Twitter CEO. The closest competitor to unlob by thesis.
- Brave Search API alternativesA genuinely independent index of 40B+ pages, sold as a conventional search API with an LLM-context option. Long the default independent choice.
- Mojeek alternativesA UK independent crawler and index, running since 2004. Small by comparison, but genuinely its own — and the oldest independent index still standing.
- Tavily alternativesAn LLM-shaped retrieval layer returning cleaned, ranked, cited chunks rather than raw SERPs. Acquired by Nebius in February 2026.
- Linkup alternativesA French grounding API returning structured, citation-carrying content for LLMs, with a sub-second "fast" tier and native parallel search.
- Valyu alternativesA London deep-search API spanning web plus licensed proprietary sources, with content attribution and pay-per-use monetisation for publishers.
- SerpAPI alternativesThe mature multi-engine SERP API. Returns real Google, Bing and Baidu result pages with all their features — at SERP-scraping prices.
- Serper alternativesThe cheapest way to get raw Google results into an agent. Fast, minimal, and entirely dependent on Google continuing to be scrapeable.
- DataForSEO alternativesIndustrial SERP infrastructure built for SEO tooling rather than agents. The cheapest per query in the field, with the lowest-level API to match.
- Google Custom Search JSON API alternativesGoogle's own programmable search endpoint. Real Google results, a hard 10,000/day ceiling, and terms written for site search rather than agents.
- Grounding with Google Search alternativesGoogle's first-party grounding for Gemini. The best index in the world, billed per search query on top of tokens, and only usable from Gemini.
- Perplexity Sonar alternativesSearch and synthesis in one call. You get an answer rather than results, which is convenient until you need to control retrieval yourself.
- Grounding with Bing Search alternativesMicrosoft's designated successor to the retired Bing Search API, available only inside Azure AI Foundry. A platform commitment, not an API swap.
- Bing Web Search API alternativesFor a decade the default independent web search API. Retired on 11 August 2025, taking Web, Image, News, Video, Entity and Custom Search with it.
- OpenAI web search tool alternativesThe built-in search tool in the Responses API. Zero integration work if you are already on OpenAI, and no control over retrieval whatsoever.
- Firecrawl alternativesCrawl, scrape and clean any URL into markdown for an LLM. A complement to a search API rather than a replacement — it needs URLs to start from.
- Jina Reader alternativesPrefix any URL and get clean markdown back. The lowest-friction extraction tool there is, and priced per token rather than per page.
- Diffbot alternativesOne of the very few independent commercial web crawls, feeding a structured Knowledge Graph. Entity-shaped rather than passage-shaped.
- Bright Data alternativesProxy and unblocking infrastructure with a scraping suite and an agent-facing MCP server. Built for volume collection, priced accordingly.
Migration guides 5
- Migration guidesParameter mappings and step-by-step ports off retired, repriced or acquired search APIs — with what you lose.
- Migrating off the Bing Web Search APIThe Bing Search API was retired in August 2025, taking Web, Image, News, Video, Entity, Autosuggest, Spell Check and Custom Search with it. Microsoft’s designated successor requires adopting Azure AI Foundry.
- The Brave Search API free tier ended — what nowBrave replaced its long-running free developer tier with a monthly credit in February 2026, so anything built on it now needs a card on file to keep running.
- Migrating from Tavily after the Nebius acquisitionTavily was acquired by Nebius in February 2026. The service continues, but acquisitions prompt re-evaluation — and Tavily never owned a web index.
- Moving off Google Search grounding to cut costGoogle Search grounding is excellent retrieval priced accordingly, and one prompt can trigger several billed queries. What the move costs, and what it saves.
Glossary 68
- Glossary
- What is Hybrid search?
- What is BM25?
- What is Semantic search?
- What is Vector search?
- What is Approximate nearest neighbour (ANN)?
- What is Reciprocal rank fusion?
- What is Embedding?
- What is Quantisation?
- What is Reranking?
- What is Query expansion?
- What is Recall and precision?
- What is Ranking?
- What is Faceted search?
- What is Filtering?
- What is Result collapsing?
- What is Admission control?
- What is Salient term?
- What is Bounded index?
- What is Index pruning?
- What is Removal ledger?
- What is Coverage transparency?
- What is Inverted index?
- What is Fast field?
- What is Resident memory?
- What is Sharding?
- What is Near-duplicate detection?
- What is Deduplication?
- What is Coverage graph?
- What is GraphRAG?
- What is Knowledge graph?
- What is Centrality?
- What is Host rank?
- What is Corroboration?
- What is Story cluster?
- What is Entity extraction?
- What is Entity dossier?
- What is Community detection?
- What is Provenance?
- What is Source lineage?
- What is Agentic search?
- What is Retrieval-augmented generation (RAG)?
- What is Context assembly?
- What is Context window?
- What is Model Context Protocol (MCP)?
- What is Tool use?
- What is Function calling?
- What is Hallucination?
- What is Chunking?
- What is Passage?
- What is Web crawler?
- What is robots.txt?
- What is Crawl frontier?
- What is Crawl-on-miss?
- What is Freshness?
- What is Boilerplate removal?
- What is Content extraction?
- What is Object-storage-first architecture?
- What is Stateless serving?
- What is Quality score?
- What is Authority?
- What is Multilingual search?
- What is Tokenisation?
- What is Rate limiting?
- What is Quota?
- What is Data residency?
- What is Sovereign AI?
- What is Evidence receipt?
Writing 2
Languages 31
- Languages
- English web search
- Spanish web search
- Chinese web search
- German web search
- French web search
- Japanese web search
- Portuguese web search
- Russian web search
- Italian web search
- Dutch web search
- Korean web search
- Arabic web search
- Hindi web search
- Polish web search
- Turkish web search
- Swedish web search
- Indonesian web search
- Vietnamese web search
- Ukrainian web search
- Czech web search
- Romanian web search
- Greek web search
- Hebrew web search
- Danish web search
- Finnish web search
- Norwegian web search
- Hungarian web search
- Thai web search
- Bengali web search
- Persian web search
Content types 14
- Content typesFilter by what a page is — assigned at index time from its structure, not guessed from the URL.
- DocumentationOfficial product, API and library documentation — reference material maintained by whoever built the thing, not third-party write-ups of it.
- CodeSource files, repository content and code-bearing pages: the implementation itself rather than an article describing an implementation.
- AcademicPapers, preprints, theses and conference writing, identified from document structure rather than from the domain that happens to host it.
- NewsReported news articles carrying a publication date and an identifiable outlet — reporting of an event, as distinct from commentary on one.
- ArticleLong-form editorial writing that is not news reporting: essays, analysis, features and engineering blogs. The general-purpose editorial bucket.
- ReferenceEncyclopedic and reference material — definitions, tables, standards and specifications, written to be looked up rather than read through.
- Q&AQuestion-and-answer pages: one stated problem and one or more proposed solutions, with the answer structure visible in the page markup.
- ForumThreaded discussion — mailing lists, message boards and community threads, where the value is in the exchange rather than in any single post.
- How-toProcedural content: step-by-step instructions for accomplishing a task, as opposed to reference material explaining how something works.
- ProductProduct and service pages, including pricing, specifications and feature listings, published by whoever sells the thing being described.
- OpinionExplicitly argumentative writing — columns, editorials and position pieces, where the author is arguing a case rather than reporting an event.
- VideoPages whose primary content is video, indexed through their text metadata and transcripts rather than through the video itself.
- DocumentStandalone documents published as files: PDFs, reports, filings and papers, where the file rather than the page is the thing you wanted.
Topics 13
- TopicsSubject tags that work across verticals.
- TechnologySoftware, hardware, infrastructure and the industry around them — assigned from what a page is about, so it holds across news, docs and papers.
- ScienceResearch findings, methods and scientific reporting across disciplines, from primary papers through to the coverage that summarises them.
- HealthMedicine, public health, clinical research and health policy — clinical material together with the policy and reporting that surrounds it.
- FinanceMarkets, banking, monetary policy and corporate finance, including the regulatory material that governs each of them.
- BusinessCompanies, strategy, funding, operations and industry structure — the organisational side of a subject rather than its technology.
- PoliticsGovernment, policy, elections and regulation, from primary legislative and regulatory texts to the reporting and analysis around them.
- EducationTeaching, learning, institutions and educational policy, across schools, universities, vocational training and professional development.
- SportsCompetition, results and the business of sport — fixtures and match reporting alongside governance, transfers and the commercial side.
- EntertainmentFilm, television, music, games and the industries behind them, covering both the works themselves and the business of making them.
- FoodCooking, ingredients, restaurants and food production, from recipes and technique through to agriculture and the supply chain behind them.
- TravelDestinations, transport, accommodation and travel logistics, including the visa, safety and regulatory material that constrains them.
- LifestylePersonal finance, home, wellness, relationships and consumer advice — the practical, personal end of a subject rather than the institutional one.