Glossary
What is Deduplication?
Verified 21 September 2026
At index time it prevents the same content occupying storage many times. At query time it prevents one item occupying every slot in a result set.
The trap is that deduplication discards information about how widely something was published — which for judging a claim is the most valuable information available. Deduplication and corroboration counting have to be designed together, or you get a compact index that cannot tell you whether anyone corroborated anything.
Common questions
What is Deduplication?
Deduplication removes redundant copies of the same content from an index or a result set, so ten slots carry ten facts rather than one fact ten times.
How does Deduplication work in practice?
The trap is that deduplication discards information about how widely something was published — which for judging a claim is the most valuable information available. Deduplication and corroboration counting have to be designed together, or you get a compact index that cannot tell you whether anyone corroborated anything.
See it working
The free tier is 10,000 credits a month with no card — enough to test any of this against your own queries.