Most legal AI is sold on demos. This page is the other thing: what we measured, on what, against what, and where the measurement stops.
Every figure below comes from a run we can reproduce. The tests were designed to make the system fail: it never sees the answer key, its output is reviewed by agents whose only job is to reject it, and the review itself is re-reviewed independently. Where a number is weaker than it looks, we say so on this page rather than waiting to be asked.
Four principles. They are the reason the numbers mean something; they are not the reason the system works.
The system is scored against material it has never seen the answers to, and every score is published together with its chance level. A 96 % result against a 14 % chance floor means what it appears to mean.
What it proves: the matching is done on function, not on recognition.
Every statement is reviewed by an independent reviewer whose brief is to reject it: find the sentence the sources do not support and fail it.
What it produced: 2 079 statements reviewed across two areas of law, 0 fabricated. Every sentence carries a source chain back to a primary source, so the verdicts are checkable too.
The review itself is reviewed, independently. That is where soft failures surface: a difference stated in the wrong direction, a materially relevant provision left out.
What it caught: direction errors fell to zero in the second pass, and quality rose between passes. Checking makes the process stricter, not softer.
A change that actually happened, a court ruling, new guidance, an amendment, is replayed against the layer as it stood before, and the layer is asked what the event touches.
What it produced: recall 1.0 against real events, and gaps human curators had missed. One CJEU ruling: all twelve countries flagged and graded, 3 high exposure, 4 to check, 4 unaffected.
Every published Homolog figure, in one table. The measure column says what the number is actually a claim about; that is usually the part that gets lost.
96–99 %
Blind reconstruction of rule equivalents
Across 24 languages. Nearly all test pairs cross-lingual. The system never sees the answer key.
80–98 %
“What is the equivalent of § X in country B?”
Measured end to end.
0
Fabricated statements in 2 079 independently reviewed
Adversarial review across two areas of law.
1.0
Change-propagation recall against real events
Deterministic. Microseconds.
2
Areas of law on the same engine
Pillar Two: 66 jurisdictions, 24 languages. GDPR opening clauses: 12 jurisdictions, 10 languages.
< 1 h
Second area of law built, validated and measured
From nothing to measured results.
The market has written the specification
Thomson Reuters, 2026 survey of tax professionals
Answered above, point for point.
| Measure | Result | What it means |
|---|---|---|
| Blind rule-equivalence reconstruction | 96–99 % | Given rules from many countries, the system reconstructs which ones do the same job; it never sees the answer key. Across 24 languages; nearly every test pair is cross-lingual. |
| Provision transposition, “what is the equivalent of § X in country B?” | 80–98 % via Homolog | The question a lawyer actually asks. Not “find me similar text”: “name the provision that does this job over there.” |
| Same question, embedding search | 17–25 % | The industry-standard approach on the same question, close to chance level. This gap is the product. |
| Fabricated statements | 0 of 2 079 | Independently, adversarially reviewed across two areas of law. Every sentence carries a source chain to a primary source. |
| Change propagation, recall | 1.0 (precision 0.90–0.95) | Against real events. Every affected assessment was found; it also surfaced gaps humans had missed. Deterministic, microseconds. |
| Worked example: one ruling, twelve countries | 3 high / 4 check / 4 unaffected | A single CJEU ruling propagated across the anchor, graded per country, with the reasoning for each grade. |
| Second domain, built end to end | 12 jurisdictions, 10 languages, under one hour | GDPR’s opening clauses: a second area of law, built, validated and measured on the unchanged engine. |
| First domain, for scale | 66 jurisdictions, 24 languages | Pillar Two. Same engine, no changes to the core. |
Ranges, and why they are ranges. Where a figure is given as a span, it is because we ran it on two different domains and are reporting both ends rather than the flattering one. Reconstruction: 96 % on the tax-law corpus, 99 % on the data-protection corpus. Transposition: 80 % on the first domain, 98 % on the second. We could quote the top of each range everywhere. We quote the range.
Chance level, stated. Reconstruction scores are always reported with the chance level for that corpus, because a percentage without a class space is not a measurement. The two corpora have different chance levels; both are in the underlying protocol.
What recall 1.0 is a claim about. Replayed against real events, every assessment the human curators had in fact updated was inside the set the layer flagged. Nothing the humans found was missed. But recall alone can be gamed: flag everything and you never miss. So it is published with its companion numbers: precision was 0.90–0.95 on those events, the flag set was 59 assessments out of 878, and the flags outside the human set were not noise. Checked afterwards, they were assessments the humans had missed. Recall says the net catches everything; the companion numbers say the net is small.
What “microseconds” is and is not. Propagation is fast because it is deterministic: the dependency structure is derived, not searched. Speed is not the claim. Recall is the claim. Speed only means a change can be propagated the moment it lands rather than in the next quarterly review.
Two kinds of comparison, and they carry different weight.
The default way to connect legal material across languages and jurisdictions is embedding search: represent the text as vectors, retrieve what is nearest. It is a genuinely good technique, and on one part of this problem it is close to us: finding the country’s rules and case law on a subject, where the texts are long and expert-written and about the same thing.
Then you ask it the question the user actually has. What corresponds to this specific provision, over there? On that question, embedding search sits close to chance level and Homolog is close to perfect. The reason is structural, not a matter of picking a better model: no embedding makes a Swedish provision reference semantically close to German statutory text. The information required to answer is not in the surface of either text.
What we ran, honestly: the baseline is a leading open multilingual embedding model, run on the identical blinded corpus, same questions, same scoring. We have not run a commercial legal-specialised embedding API against it. Our reading of the field is that it would not change the outcome: the failure class here is cross-lingual confusion between neighbouring legal concepts, and the closest legal-specialised candidate is English-trained without a multilingual claim. But we have not measured it, and we will not claim it. If a partner wants that number, we will run it.
Where the baseline actually loses. Not randomly. The misses cluster: a rule about who must appoint an officer gets matched to a rule about health data; a rule on journalism gets matched to a rule on research. Same subject area, different function: who is obliged to do what. That is the distinction the layer is built on, and it is the distinction similarity cannot see.
There is no shipping product that does this, so there is no product benchmark to quote. The nearest published work measures adjacent things, and the comparison is directional only:1
The footnote is not decoration. Read those as context for the order of magnitude, not as head-to-head results.
1 Different tasks, different corpora, different scoring. Our reconstruction test scores against a defined anchor set rather than open retrieval, which is an easier setting than open-web search; our reviewers check claims against cited sources rather than practising in the jurisdiction. Anyone quoting these side by side without those caveats is overselling, including us, if we did it. The honest summary: on the axes where a published comparison exists at all, we are at or above the published front; on the two axes that are the actual product, transposition and propagation, no comparison exists, which is the reason we intend to build the benchmark.
The strongest evidence that something is a technology and not a dataset is that it survives a change of subject.
| Pillar Two | GDPR opening clauses | |
|---|---|---|
| Area of law | International tax: calculation-driven | Data protection: rule-driven, no computation |
| Jurisdictions | 66 | 12 |
| Languages | 24 | 10 |
| Cross-lingual test pairs | Substantial | Nearly all |
| Blind reconstruction | 96 % | 99 % |
| Provision transposition | 80 % | 98 % |
| Change propagation | recall 1.0 | recall 1.0 |
| Fabrications found | 0 | 0 |
These two legal families have almost nothing in common. One is arithmetic wearing a statute; the other is obligations, thresholds and consequences with no formula anywhere. Their vocabularies are effectively disjoint: of the concepts each domain produced, the two share a handful of terms, all of them from the shared administrative-procedure core. That is the useful finding: domains add up rather than collide, so a second domain does not disturb the first.
The second domain scored higher than the first. We are not going to present that as a trend from two data points. The data-protection corpus is smaller and its anchor set is narrower, which makes the reconstruction task easier. We say that in the protocol and we say it here. What the run does establish is the one thing it was built to establish: the engine did not need to change. A reader for the new corpus format and two domain-specific fields. Nothing in the graph, the matching, the difference derivation or the ontology.
Built, validated and measured in under an hour. That is the honest cost picture for adding a domain once the anchor exists, and the anchor is the real variable, not the engine. Areas of law with a strong shared anchor (EU regulations and directives, OECD model rules, standards) are where this runs today. Areas without one are a harder problem, and we would rather tell you that than sell you into it.
The methodology is the product. Full protocols are available under NDA.
What is on this page is what each test proves and what it scored. What is not on this page: how the blinding is constructed, what the reviewers are actually asked, how the layer is built, how differences are derived, what the extraction produces. Those are the machine. Publishing them would hand a competent competitor the build, and the reason a partner works with us rather than around us is that we operate the machine rather than licensing it.
Under NDA, a serious counterparty gets the full protocol: the test design, the scoring rules, the failure analysis including the cases we lost, the raw review verdicts, and the runs reproduced on material you supply. Everything on this page is checkable at that table. We would rather be examined properly once than argued with publicly forever.
Partner and client data is never used to train models. The layer is retrieval and reference, with verifiable sources on every statement. Your material stays yours, and the process runs in our house.
The benchmark we intend to build. There is no public test set for functional equivalence of rules across jurisdictions and languages: that is why this page compares against research rather than against competitors. We are building one: an open evaluation set for the transposition question and for derived difference, so that this category can be measured by someone other than the people selling into it. Whoever holds the measuring stick holds the argument, and we would rather that stick were public.