Measured against real sales.

How the model is tested - and how wrong it tends to be.

We hold out a fixed set of recorded sales - the same domains every run, never part of the engine’s evidence - price them blind, and publish the error for the active methodology and for any candidate revision before it can become the default. A candidate ships only if it beats the incumbent here. Behind it sits a recorded-sales ledger spanning three decades of the aftermarket.

Two sales figures appear on this site and they are different things: the full comparable corpus of curated recorded sales that the engine queries for comparables, and the held-out back-test set - recorded sales kept out of that corpus, so the engine never sees the prices it is scored against. The headline figures on this page are scored against the 218 recorded sales held out of the engine's evidence for back-testing.

Active: revision 4.14. Structural pricing is revision 4.13; 4.14 adds only the personal-name path - every non-name class is identical to 4.13.

Measured over 218 recorded sales held out of the engine's evidence for back-testing, so the engine never saw the prices it is scored against. This is the conservative figure, and the one to plan against. Revision 4.13 is the active structural methodology; revision 4.14 is identical to it for every class except recognized personal names, where it adds a separate path on top. The full comparison against the retired revision, which never saw these sales either, is below.

The headline is the recent sales cut - sales reported by DNJournal and Sedo, kept out of the corpus the engine draws its evidence from - one of two we measure on the same frozen test set. It was chosen for where its prices come from, not for how the model scores on it, although it is also the cut the model scores best on. The model is weakest on the bulk ledger sample cut. The bulk ledger sample stays published because it is the one closest to screening a drop feed, which is what the numbers get used for. Read the headline as the accuracy on names like the ones in its cut, not as one figure that covers every name.

One estimator (champion (rev 4.13, generalization)), one frozen leakage-guarded test set of 618 names, two disjoint slices; no name is scored twice. The published row is the same measurement as the headline above. Test set 5866aaffbc4e, scored in tools/scoreboard/baselines/champion.json. The curated holdout cut is no longer scored: the 2026-09-23 provenance audit withdrew 171 of its sales as generated or unrecorded, leaving 3, too few to measure.

The smaller figure under each number is its 95% interval - the range the sample size can actually support. Read the interval, not the decimal place.

Median error and median error as a factor measure the same misses two ways, and the second is the one to trust. A percentage error cannot go below -100% but has no ceiling above, so on prices - which move by multiples, not by percentages - the same miss scores twice as badly when the estimate lands high as when it lands low. A 2× underestimate scores 50%; a 2× overestimate scores 100%; a 10× underestimate scores 90%, which reads better than the mild 2× overestimate it is far worse than. The factor column removes that tilt by measuring the miss in whichever direction it falls, so the published 5.0× means half of all estimates land within 5.0× of the eventual sale price - above or below.

Every figure above asks how close a price is. This asks a different question: given a list, does the engine put the right names at the top? For anyone triaging a drop list or a portfolio, that ordering is the thing being bought, and it is measured on the same frozen test set, slice by slice.

Rank correlation is Spearman's: 1.00 would be a perfect ordering, 0.00 no better than shuffling. Top 50 correct is how many of the fifty names the model puts first really do belong in the slice's top fifty. Value in top 50 is the share of everything that slice eventually sold for which those fifty names account for - the figure that decides whether triaging by this order is worth doing.

The slices are reported separately and not pooled. A pooled figure would read higher than any row here, because the slices differ in price by orders of magnitude and most of a pooled ordering is decided by which slice a name came from rather than by any judgement about the name. Scored in tools/scoreboard/baselines/two-output.json against test set 5866aaffbc4e, the same one the price table above uses.

How accurate is the audit.domains appraisal, and did the methodology improve?

Measured on 218 recorded sales, each sale withheld from the engine's evidence, evaluated September 30, 2026: the live methodology (revision 4.14) has a median error of 85.1% and lands within 2× of the recorded price 29.4% of the time, within 3× 39.9% of the time. The revision it replaced (4.2, run as shipped, never having seen these sales, the same 218 sales) had a median error of 99.8% and was within 2× 18.8% of the time. Every revision is below, including the two that carry no figure because they have not been measured.

Machine-readable copy: /accuracy/backtest.json carries this table with each figure’s cut label and basis attached, generated from the same committed facts as this page, and fetchable from any origin.