Skip to content
unlob
Get an API key

unlob for LlamaIndex

The two-step shape suits LlamaIndex well: retrieve metadata cheaply, select, then load the full text of only what survived selection.

A LlamaIndex retrieverpython
import os, requests
from llama_index.core.retrievers import BaseRetriever
from llama_index.core.schema import NodeWithScore, TextNode

BASE = "https://api.unlob.com"
HEADERS = {"x-api-key": os.environ["UNLOB_API_KEY"]}

class UnlobRetriever(BaseRetriever):
    def __init__(self, top_k: int = 8, min_sources: int = 2):
        self.top_k, self.min_sources = top_k, min_sources
        super().__init__()

    def _retrieve(self, query_bundle) -> list[NodeWithScore]:
        r = requests.get(f"{BASE}/search", headers=HEADERS, timeout=10,
            params={"q": query_bundle.query_str, "limit": self.top_k,
                    "collapse": "story",
                    "min_independent_sources": self.min_sources})
        r.raise_for_status()
        return [
            NodeWithScore(
                node=TextNode(text=h["snippet"], id_=h["id"],
                              metadata={"url": h["url"], "host": h["host"],
                                        "sources": h["independent_sources"]}),
                score=h["score"],
            )
            for h in r.json()["results"]
        ]

Setup

  1. Get an API key

    10,000 free requests a month.

  2. Implement the retriever

    Subclass BaseRetriever and map hits onto nodes. Keep independent_sources in the metadata — it is a better trust signal than the relevance score.

  3. Load text lazily

    Call /doc/:id only for nodes that survive your postprocessors, rather than fetching everything up front.

  4. Consider skipping the pipeline

    If your chain is retrieve-dedupe-rerank-truncate, assemble_context does all of it server-side and returns a reason per passage.

Worth knowing

  • Snippets are usually enough for the selection step. Fetching full text before selection is the most common source of wasted tokens in a RAG pipeline.
  • independent_sources in node metadata lets a postprocessor filter on corroboration, which relevance scores cannot express.

Frequently asked questions

Should I use the retriever or assemble_context?

The retriever when you have your own reranking and postprocessing you trust. assemble_context when you would rather not maintain that pipeline — it does the same work server-side and explains each inclusion.

Get a key and connect it

10,000 requests a month, no card. The config above works unchanged once you have a key.

Get your API keyAPI reference