Test-Set Leakage: The Quiet Reason Results Look Better Than Reality

The question

A model scores 87% on a benchmark. The benchmark is public. The model was trained on data scraped from the internet. How much of the 87% is capability, and how much is the model recognizing the exam?

That question is usually answered with a detection technique — n-gram overlap, canary strings, post-cutoff performance decay — and the technique usually produces a binary: “contaminated” or “not.” Both answers are less useful than they appear. Test-set leakage is not a single failure with a single detector. It is at least three distinct mechanisms, they inflate results in different ways, and a clean detection result for one mechanism says nothing about the other two.

This post separates the three, quantifies what the literature says about each, and states the reporting discipline that keeps a leaked result from becoming a decision.

The short answer

Leakage reaches a benchmark score through three independent channels:

They are not the same problem, and the most commonly discussed one — contamination — is not the most consistently inflating one. A controlled study across 2,047 benchmark datasets found that selection leakage, particularly peeking, was the most universal threat at practical dataset sizes (AUC = +0.040, 92% prevalence), while the per-fold preprocessing leakage that most textbooks emphasize was negligible (AUC < 0.005). The pedagogical emphasis and the actual risk are misaligned.

The rest of this post is each mechanism, its evidence, and what a report has to say to keep the score interpretable.

Contamination: real, but dose-dependent

The standard story is that test items in pretraining data inflate scores through memorization. That story is true and incomplete, and the incompleteness is why “contaminated” is a less useful label than it sounds.

The NeurIPS analysis of the contamination paradox resolves the apparent contradiction between work showing contamination is potent and work showing it is negligible in the “overtrained” regime common to frontier systems. The resolution is a dose-response framework: a model memorizes a benchmark based on its capacity (number of parameters) and its incentive (the relative training loss reduction from memorizing test data). Contamination is not a switch. It is a dose mediated by model size, and the effect scales with the proportion of benchmark tokens in the pretraining corpus.

The ICML 2025 controlled study on machine translation makes the dose concrete. Starting from a decontaminated split, reintroducing test examples produced BLEU inflation up to 30 points for an 8B model and about 8 points for a 1B model — the effect was 2.5× larger at the larger scale. Leaking only the source or only the target produced smaller, inconsistent over-estimations; leaking both source and target together was the potent configuration.

Two consequences follow for practice. First, the same contamination dose can be negligible for one model and material for another, so a contamination rate is not a universal score adjustment. Second, the configuration of the leak matters — a benchmark whose items appear with solutions and reasoning traces is a different exposure than one whose items appear as bare questions.

The multi-method audit of six frontier models on MMLU gives a scale for the contamination itself: an overall lexical contamination rate of 13.8%, with 18.1% in STEM and up to 66.7% in Philosophy, and estimated performance gains of 0.030–0.054 accuracy points by category. The same audit found that paraphrasing the questions dropped accuracy by an average of 7.0 percentage points, rising to 19.8 points in Law and Ethics — precisely the most-contaminated domains. A model that has memorized the exam answers the memorized form and is brittle on the paraphrase, and the benchmark score is computed on the memorized form.

The rule: report the contamination mechanism, its dose, and the model scale it was measured at, or report the score as unadjusted for contamination. A bare “contaminated: yes” is not a usable correction.

Selection leakage: the one the score cannot show

Selection leakage is the mechanism that produces inflated results without any test item appearing in training data. It works through the evaluation process itself: the test set is used to make a decision that affects the model or configuration being evaluated, and the test set stops being held out.

The controlled leakage study quantifies the effect and the process: peeking at test performance during model selection produced AUC = +0.040 at 92% prevalence, and seed cherry-picking produced ΔAUC = +0.045. The mechanism decomposes into a diversity component (genuine performance differences across the selection pool) and a noise component (exploitation of estimation variance). The noise term follows extreme value theory — roughly σ√(2 ln K), where σ is the standard error of the estimate and K is the number of configurations examined.

That formula is the practical warning. The inflation grows with the number of configurations compared, which means the standard engineering practice of sweeping several options and picking the best is itself a leakage mechanism. A single configuration evaluated once on the test set may show no inflation; ten configurations evaluated and the best selected will show an inflated best, and the inflation is invisible in the winning score.

The MLOps convention states the consequence directly: the evaluation set is held out permanently — updating it invalidates the metric. Adding data to the test set after seeing what the model got wrong is still leakage, because the new data was selected with knowledge of the failures.

The rule: selection leakage cannot be detected from the score. It can only be prevented by process: a single held-out evaluation, a pre-registered selection procedure, and a record of how many configurations were compared. A report that does not state K has not ruled out this mechanism.

Temporal leakage: the cutoff check is not enough

Temporal leakage is the mechanism that is easiest to mistake for something else, because the standard detection check is uninformative by construction.

The standard check compares scores before and after the training cutoff. The 2026 analysis of temporal leakage in LLM backtesting shows why this fails: four flagship models failed the cutoff check on questions they could not have memorized, every scored question having resolved after their cutoffs. The reason is structural — models legitimately know more about times near their cutoff, so recency mimics leakage, and no passive backtest can separate the two from genuine skill.

The practical mitigation requires information from outside the backtest: a known cutoff to localize leakage at the boundary, and a matched clean control to estimate it globally and produce a leakage-adjusted score. The analysis also locates where the leakage hides: it concentrates on outcomes that surprised the crowd and were well covered in training, and partial memorization is disproportionately rewarded.

The claim-level framework adds a measurable unit: decomposing model rationales into atomic claims and computing a Shapley-weighted Decision-Critical Leakage Rate — the fraction of decision-driving reasoning that derives from leaked information. On 350 instances across Supreme Court prediction, NBA salary estimation, and stock return ranking, standard prompting baselines showed substantial leakage.

The rule: a temporal split is not a clean split unless the evaluation uses a matched control or a defensible reference. “We compared before and after the cutoff” is not evidence that leakage is absent; it is evidence that the check was run.

The Argon case: a harness difference that looks like a capability difference

The ARC-AGI-3 result is the clearest public example of a score difference that a leakage frame cannot explain alone, but that a reporting frame must.

gemini-4-argon reached 97.5% on ARC-AGI-3 through a provider-adapter harness that uses retained reasoning and context compaction, at a reported cost of $4,800**. The same model on the same puzzles reached 49.8% through the standard harness that everyone else uses, at a reported cost of **$5,800.

The harness is not a contamination mechanism. It is a measurement-condition difference, and it is included here because it is the case a reader is most likely to misattribute. A model that retains reasoning across rounds and compacts context is not seeing leaked answers; it is being evaluated under a different operational definition of “solving the task.” The provider-adapter score and the standard-harness score are measurements of different conditions, and comparing them to another model’s standard-harness score is a comparison between a condition and a condition — not between a model and a model.

The related reporting problem is the aggregate. Of the 19 benchmark rows in the release table, 10 were calculated by the provider itself and 9 came from public leaderboards, with different harnesses per row, and access was limited to a defender program at the time — so the numbers could not be independently reproduced outside the provider’s partners and a small set of benchmark organizations.

The lesson is not that the 97.5% is wrong. It is that the score is a measurement under a stated harness, and the harness is part of the measurement. A leakage audit that checks contamination and ignores harness difference has audited one of the ways a score can exceed reality and missed another.

What detection can and cannot do

The detection toolbox is real and it is incomplete, in a specific way.

N-gram overlap is the oldest and most widely deployed method, using ranges from 8-grams to 13-grams. It is trivially defeated by paraphrasing — work cited in the detection literature shows that paraphrased examples preserve full memorization while evading all n-gram filters. The 7.0–19.8 point accuracy drop on paraphrased MMLU questions is the empirical version of this: the memorization survives the paraphrase, and the detector does not.

Canary strings detect whether a specific marker sequence was in training data. They are useful for tracking a known benchmark's exposure and weaker as a general-purpose detector, because a model can be contaminated by items that were never canaried.

Likelihood-based probes infer membership from the model’s own likelihoods. They are model-internal and do not require the training corpus, and their power depends on the memorization signature being distinguishable from fluent generation — which, per the six-model audit, it often is not: 72.5% of questions triggered memorization signals, with one model showing 76.6% partial reconstruction and 0% verbatim recall, a signature that a verbatim-recall detector would miss entirely.

The structural point: detection estimates contamination, and contamination is one of three mechanisms. A clean detection result is a statement about one channel. It is not a statement that the score is unadjusted.

The record that makes a score interpretable

A benchmark result is citable when it carries the fields that let a reader assess all three mechanisms. The minimum:

json

{
  "benchmark_id": "...",
  "benchmark_public_since": "2024-06-01",
  "model_training_cutoff": "2025-09-01",
  "n_items": 400,
  "score": 0.87,
  "contamination": {
    "check_performed": true,
    "method": "n_gram_overlap_13",
    "contamination_rate": 0.0,
    "dose_response_estimated": false,
    "paraphrase_check": {"performed": true, "score_drop_pp": 7.0}
  },
  "selection": {
    "configurations_compared": 5,
    "held_out_evaluation": true,
    "test_set_updated_after_results": false,
    "selection_procedure_preregistered": true
  },
  "temporal": {
    "split_type": "random",
    "matched_control_present": false,
    "cutoff_comparison_only": false
  },
  "harness": {
    "harness_id": "standard_v1",
    "provider_specific_features": false
  },
  "adjusted_score": null,
  "not_tested": ["temporal leakage beyond the cutoff boundary", "partial-memorization probes"],
  "evidence_level": "reproducible experiment"
}

Four field groups do the work:

• contamination.dose_response_estimated separates “we checked for exact overlap” from “we estimated the effect at this model scale.” The contamination paradox says those are different claims.

• selection.configurations_compared and held_out_evaluation are the only available evidence against selection leakage, because the score cannot show it. A report with five configurations compared and a non-preregistered procedure has an upward-biased best.

• temporal.matched_control_present states whether the temporal check was a cutoff comparison (uninformative) or a controlled comparison (informative).

• harness.provider_specific_features states whether the score is comparable to a standard-harness score. The ARC-AGI-3 case is the reason the field exists.

adjusted_score is deliberately nullable. A leakage-adjusted score requires an estimate of the leakage effect, and most reports do not have one. Null is an honest value; a score presented without adjustment and without a leakage caveat is not.

Failure cases observed

The overlap-clean benchmark. An n-gram check found no exact overlap. The benchmark items had been paraphrased in a public dataset that was in the training corpus. The contamination was real and the detector was defeated by the paraphrase. The accuracy drop on a rephrased evaluation was 6.8 points, and it was measured only after the decision had been made.

The best-of-five. Five configurations compared on the test set, the best selected, the winner reported. The reported gain was within the noise band the selection procedure was designed to exploit. The test set was not updated; it did not need to be. The selection was the leak.

The cutoff check. A temporal benchmark compared scores on pre-cutoff and post-cutoff items, found no significant difference, and reported the evaluation as clean. The cutoff check was structurally incapable of detecting the recency effect, and the four flagship models in the 2026 analysis would have passed the same check on questions they could not have memorized.

The partial-memorization model. A model with 76.6% partial reconstruction and 0% verbatim recall passed a verbatim-recall contamination check and failed a paraphrase check. The detector was measuring the wrong memorization signature for that model.

The harness comparison. A local score was compared against the ARC-AGI-3 provider-adapter figure, treating the harness difference as a rounding detail. The comparison was between two measurement conditions, and the report presented it as a comparison between two models.

None of these are model failures. All of them are leakage or measurement-condition failures, and each maps to a mechanism the reporting discipline is designed to surface.

What this post does not prove

It does not prove that every benchmark is contaminated. A private, recently constructed, never-published evaluation set can have low contamination, and the contamination literature’s dose-response framing implies that a benchmark with no exposure has no contamination effect by construction.

It does not prove that leakage explains every inflated score. Harness differences, scorer instability, and construct invalidity inflate scores through non-leakage channels, and the accuracy/validity/completeness/usability post is the relevant frame for those.

It does not resolve the contamination paradox for a specific model. The dose-response relationship depends on capacity and incentive, and the parameters are model-specific; the post states the framework, not a universal correction factor.

It does not claim that the temporal-leakage controls are always available. The 2026 analysis explicitly notes that measurement requires information from outside the backtest, and that information is not always obtainable. The honest report in that case says the temporal channel was not measured.

What it does establish is the structure of the problem: test-set leakage is three mechanisms, only one of which is detectable from the data, and the score cannot adjudicate any of them without process and condition fields. The reporting record is the correction. It is more work to write than the score, and it is the only part of the report that lets a reader decide whether the 87% means what the decision assumes.

A working demo is evidence. It is not a production system. A clean contamination check is evidence about one leakage channel. It is not evidence that the score is unadjusted — and the distance between those two statements is where the quiet inflation lives.

Reproduction notes

• Contamination paradox and dose-response framework referenced from the NeurIPS analysis: memorization depends on model capacity and incentive, mediated by the proportion of contaminated pretraining tokens.

• Machine translation contamination figures referenced from the ICML 2025 controlled study: up to 30 BLEU points inflation for 8B vs. ~8 points for 1B; source-and-target leakage the potent configuration.

• Leakage-type prevalence figures referenced from the quantitative landscape across 2,047 benchmark datasets: selection peeking AUC = +0.040 at 92% prevalence; seed cherry-picking ΔAUC = +0.045; per-fold preprocessing AUC < 0.005; noise term follows extreme value theory with σ√(2 ln K) growth in the number of configurations.

• MMLU multi-method audit figures referenced: 13.8% overall lexical contamination, 18.1% STEM, up to 66.7% Philosophy; 0.030–0.054 accuracy-point estimated gains; paraphrase accuracy drop 7.0 pp average, 19.8 pp in Law and Ethics; 72.5% memorization-signal rate; 76.6% partial reconstruction / 0% verbatim recall for one model.

• Temporal leakage findings referenced: cutoff comparison uninformative; four flagship models fail it on unmemorizable questions; recency mimics leakage; matched clean control required for global estimation; leakage concentrates on crowd-surprising, well-covered outcomes; Shapley-DCLR claim-level metric.

• Detection-method limitations referenced: n-gram overlap 8–13 grams, defeated by paraphrase preserving memorization; canary strings; likelihood-based probes.

• MLOps held-out rule referenced: evaluation set held out permanently; updating it invalidates the metric; post-hoc additions are selection leakage.

• ARC-AGI-3 harness comparison referenced: 97.5% provider-adapter at reported $4,800 vs. 49.8% standard harness at reported $5,800; 19 benchmark rows with 10 provider-calculated and 9 from public leaderboards, different harnesses per row; access limited to a defender program at release.

• Interval and resolution conventions referenced from prior posts in this series: test-set construction, small-benchmark misunderstanding, comparison reporting.

For AI tool features, rules and availability, refer to their official sites.