Use case
Serving users in many languages
Teams whose users search and read in different languages.
The problem, and the approach
The problem
- Per-language indexes multiply infrastructure and keep each language's users inside their own corpus.
- Translating queries loses meaning at the boundary and doubles latency.
- Coverage is genuinely uneven across languages, and most stacks discover this only when a user complains.
The approach
- A single multilingual embedder places every supported language in one space, so a query in one language retrieves relevant passages in any other.
- The lang filter pins output language when the user needs one, and leaving it unset gives the best sources regardless.
- Facets on language show coverage distribution before you answer, so thin coverage is something you can state rather than something the user discovers.
The parameters that matter
| Parameter | Value | Why |
|---|---|---|
mode | semantic or hybrid | Keyword mode is monolingual by construction. |
lang | user’s language | Pin when the answer must be readable to them. |
facets | true | See language distribution before answering. |
tld[] | country codes | Approximate jurisdiction alongside language. |
# German query, sources in any language
curl -H "x-api-key: $UNLOB_API_KEY" "https://api.unlob.com/search?q=Klimapolitik&mode=semantic"
# Pinned to German sources
curl -H "x-api-key: $UNLOB_API_KEY" "https://api.unlob.com/search?q=Klimapolitik&lang=de"What a single embedding space removes
The conventional multilingual stack is a translation step in front of a per-language index. It has two costs that compound: infrastructure multiplied by the number of languages, and meaning lost at the translation boundary — the query that gets executed is a translation of the question rather than the question, and the failure is invisible because something always comes back.
Placing every language in one space removes the step rather than improving it. A question asked in Portuguese retrieves a Japanese passage because the passage is about the thing that was asked, and no translation was involved in deciding that. For a product whose users read one language and whose best sources are written in another, this is the difference between an answer and an apology.
Coverage is not parity, and the facets will say so
One embedding space does not mean one volume of material. English is by far the largest slice of the open web and therefore of any index over it, and a cross-language query will quietly lean on it. That is often the right outcome and occasionally a serious one — a question about local regulation answered from English-language secondary coverage is a worse answer than a thinner set of primary sources in the language it was written in.
The practical habit is to look before assuming. A cheap probe returning counts and language distribution with no results shows what is actually available in each language, which lets the product say coverage is thin here rather than silently returning weaker sources. Pin the language when the primary source must be in it; leave it unpinned when the meaning is what matters. And note the tokenisation boundary: character-bigram handling for CJK and Thai is robust but occasionally less precise than a language-specific segmenter, which shows up on short queries more than long ones.
Where this is not the right tool
- Coverage is uneven across languages — English is by far the largest slice. Check the facets rather than assuming parity.
- Character-bigram tokenisation for CJK and Thai is robust but occasionally less precise than a language-specific segmenter.
Frequently asked questions
Do I need to translate queries?
No. Semantic and hybrid modes cross languages directly, and translating would search for the translation rather than the meaning.
How many languages are supported?
101 in the embedding space. The lang filter accepts the full ISO set, and the 30 highest-volume languages have their own pages with sample queries.
Build it on the free tier
10,000 credits a month, no card. Every parameter above works on every plan.