With Recansio, and without.
Two tests on real legal questions, each with its method and its limits.
Tokens, time and precision.
24 questions, 16 November 2026.
Questions written by a practising lawyer, with the lawyer's answers. One model (Claude Opus 5.5) answered them twice: once with web search only, once with Recansio only. Three AI graders scored both answers against the lawyer's, in random order and without being told which side wrote which. They saw the text of every paragraph an answer linked to.
fewer tokens than pasting the acts for each question
- With Recansio
- 9,500 a question
- Uploading the acts
- 131,000
- Web search, as counted
- 7,500
faster than searching the web
- With Recansio
- 18 s a question
- Web search
- 62 s
- All 24
- 7.3 min / 25.0 min
answers rest on the exact paragraph
- With Recansio
- 20 correct, 24 exact
- Web search
- 21 correct, 1 exact
| Question, by area | With Recansio | Web search |
|---|---|---|
| 1. Anti-money-laundering · asked in Uzbek | correct | correct |
| 2. Anti-money-laundering · asked in English | correct | correct |
| 3. Banking · asked in Russian | partly | correct |
| 4. Banking · asked in Russian | partly | correct |
| 5. Banking · asked in Uzbek | correct | correct |
| 6. Civil law · asked in Russian | correct | correct |
| 7. Civil law · asked in Russian | correct | correct |
| 8. Civil law · asked in Russian | correct | correct |
| 9. Criminal and administrative liability · asked in English | correct | partly |
| 10. Criminal and administrative liability · asked in Russian | correct | partly |
| 11. Criminal and administrative liability · asked in Uzbek | correct | correct |
| 12. Criminal and administrative liability · asked in Uzbek | correct | correct |
| 13. Criminal and administrative liability · asked in Uzbek | correct | correct |
| 14. Insurance and capital market · asked in Uzbek | correct | wrong |
| 15. Insurance and capital market · asked in Russian | wrong | correct |
| 16. Lending · asked in Uzbek | correct | correct |
| 17. Lending · asked in Uzbek | partly | correct |
| 18. Labour and tax · asked in Uzbek | correct | correct |
| 19. Labour and tax · asked in Uzbek | correct | correct |
| 20. Labour and tax · asked in Uzbek | correct | correct |
| 21. Payments and currency · asked in Russian | correct | correct |
| 22. Payments and currency · asked in Russian | correct | correct |
| 23. Payments and currency · asked in Uzbek | correct | correct |
| 24. Public law · asked in Uzbek | correct | correct |
- How tokens were counted
- The whole run of the assistant for each batch of six questions, minus what an empty run costs (35,875 tokens), divided by six. For uploading: the Russian text of the acts each answer rests on, read from the base, at 0.454 tokens a character, which we measured by having the same model read 119,824 characters of the Labour Code. The middle value across the 24 questions is shown; 11 of them rest on two acts or more. Uploading all 29 acts once would take 2.9 million tokens.
- What the web-search count leaves out
- The pages the assistant opens are read by a separate tool, and those tokens are not in its count. So its own count (7,500 a question) is close to Recansio's, while the work took 3.4 times as long.
- What "exact" means
- At least one paragraph the answer links to itself states the deciding rule. "Near" means a link to the right act or a neighbouring part. The graders saw the text of every linked paragraph, read from the base.
- Limits
- 24 questions: a difference of one or two answers is noise. The graders are models, not lawyers; the three agreed on 24 of 24 verdicts for Recansio and 22 for web search. If all your questions sit in one act, you upload it once per conversation, not once per question. The question texts stay unpublished so the set can be used blind again.
An earlier test: five assistants, 30 and 42 questions.
A strong assistant with web search can already answer correctly: ChatGPT got 30 of 30 on its own, and 28 with Recansio. What no assistant did on its own was show where the law says so. Of 180 answers given without Recansio (Claude 12, the other four 42 each), none linked to the exact paragraph. With Recansio, 118 of 126 did. That is what Recansio is for: an answer you can check.
Exact paragraph cited
Answers that link to the paragraph of the law, not to the act or a news site. Claude on its own was measured on 12 answers, the others on 42.
Correct answers
30 client questions in three languages.
Time
From sending the questions to the final answer. Less is better.
A free model on a laptop
Qwen 27B, offline, 64 GB of memory.
| On its own | With Recansio | |
|---|---|---|
| Correct, 30 client questions | 5 | 24 |
| Correct, 12 hard questions | 1 | 4 |
| Wrong, 30 client questions | 15 | 2 |
| Cite the exact paragraph, 42 answers | 0 | 40 |
In practice
Product risk assessment in a bank. Seen in use, not a controlled test.
Method and limits
- Questions
- 30 that clients asked, and 12 hard ones: past dates, appendices, licences, sanctions.
- Grading
- Three graders, each blind to which system wrote the answer. Majority verdict.
- Limits
- Small sets, one run each. One wrong answer moves a score by 3 to 8 points.
Try it on your own questions.
Test access opens as soon as you sign up: 100 calls over 7 days, no payment details.