Use case
Fact-checking with corroboration counts
Teams that must verify claims before repeating them.
The problem, and the approach
The problem
- A ranked result list cannot distinguish twelve copies of one wire story from six independent reports, and it presents the twelve copies as more convincing.
- Naive deduplication makes this worse: collapsing forty copies to one destroys the evidence of how widely the claim was carried.
- Verifying by hand means cross-searching and manually assessing source independence, which is exactly what an agent does badly.
The approach
- independent_sources counts distinct asserting hosts, and the count survives deduplication because a rejected duplicate's host is recorded against the surviving passage.
- corroborate returns the source groups directly, with the number of merged duplicates alongside — a story with two sources and thirty duplicates is a press release; two with none is two newsrooms.
- min_independent_sources applies a threshold uniformly, so it is a policy you can document rather than a judgement made per query.
The parameters that matter
| Parameter | Value | Why |
|---|---|---|
min_independent_sources | 3 | A defensible floor for anything repeated to a user as fact. |
collapse | story | One representative per cluster, so the result set is distinct claims. |
min_host_rank | 0.5 | Removes low-authority publishers from the corroboration count. |
corroborate?id= | the passage | The direct check for a specific claim. |
curl -H "x-api-key: $UNLOB_API_KEY" \
"https://api.unlob.com/search?q=acquisition+announced&min_independent_sources=3&collapse=story"
curl -H "x-api-key: $UNLOB_API_KEY" "https://api.unlob.com/corroborate?id=p:8f2c9a"Counting hosts is not the same as counting sources
Corroboration is only as good as the independence assumption behind it, and on the open web that assumption is routinely violated in two directions. Several outlets carrying the same wire copy look like several sources and are one. A single outlet running a story, a follow-up and an analysis piece looks like three and is one newsroom.
Deduplication is what keeps the first case honest: near-duplicate detection folds the syndicated copies into one cluster, and the count of merged duplicates is reported separately so you can see that the apparent breadth was syndication. It does not settle the second case, and nothing automatic will — the judgement about whether two outlets share a reporting pipeline is editorial, and a verification workflow that pretends otherwise is producing a number that reads as evidence and is not.
What a corroboration threshold is actually for
A threshold does not decide whether a claim is true. It decides what reaches the model, and its value is that it moves a judgement out of the prompt, where it is unreliable and invisible, into the query, where it is explicit and reviewable.
That has a consequence worth stating plainly: a threshold of three will suppress a true story that has been broken by one outlet and not yet followed. On a breaking story that is the wrong behaviour, and the right response is to lower the threshold deliberately and report the reduced corroboration alongside the answer rather than to remove the filter and forget it was there. The failure mode a threshold prevents — twelve copies of one unverified claim read as consensus — is more common and more damaging than the one it causes, but both are real.
What a fair test looks like
A representative task: Decide whether a specific claim is supported, contradicted or unverifiable. Take twenty of yours, write down what a good result is before running anything, and score them on:
- Verdicts match a reviewer’s on a labelled set, including the unverifiable ones.
- Syndicated copies do not count as independent corroboration.
- Absence of evidence is reported with why_not, not read as a refutation.
Include tasks where the right answer is to abstain, and report the cases where unlob does not fit alongside the ones where it does. How an evaluation works.
What it costs
The recipe above, as an illustrative call pattern: 1 search + 2 corroborate + 1 authoritiesper claim checked — 7 credits. Find the claim, count independent hosts behind it and behind its strongest rebuttal, and check whether an authority has spoken.
| Tasks a month | Credits | Cheapest plan | Per 1,000 tasks |
|---|---|---|---|
| 1,000 | 7,000 | Build, $29/mo | $29 |
| 10,000 | 70,000 | Build, $49/mo with overage | $4.90 |
| 100,000 | 700,000 | Scale, $319/mo with overage | $3.19 |
Provider spend only; your model and review effort are separate lines. The pattern is a starting point, not a measurement — cost your own.
Where this is not the right tool
- Independent hosts are not necessarily independent reporting — several outlets may share a wire service. The merged_duplicates count helps distinguish these but does not settle it.
- Corroboration measures how widely a claim is carried, not whether it is true. Widely repeated falsehoods exist.
Frequently asked questions
What threshold should I use?
Two as a general floor and three for anything stated to a user as fact. Higher thresholds trade coverage for confidence, and the right point is worth measuring on your own queries.
Does this detect misinformation?
No. It measures how many distinct sources assert something, which is a useful signal and not a truth judgement. A widely syndicated falsehood scores well on corroboration.
Evaluate it on twenty of your tasks
Send the tasks and what a good result looks like, and we report what was found, what was missing, the cost and where unlob does not fit — or run it yourself on 10,000 free credits a month.