SONDE The Standard

Six rules, enforced by the schema rather than by review

Each rule exists because of a published result about how biological benchmarks mislead. A result card that does not carry its baseline, its reference and its null does not validate, so the failure mode is a crash at emission time, not a misleading number on a leaderboard.

None of these rules is ours. Splitting by sequence similarity, requiring a no-model baseline and resampling whole groups rather than individual items are long-standing practice in the established protein and single-cell benchmarks, and several of them do it more strictly than we currently manage. The examples given under each rule below are drawn from our own papers because those are the failures we know first-hand, not because we found the problems first. What is unusual here is only that the schema enforces the rules instead of trusting a reviewer to notice. The prior-work page gives the credit properly.

Beat a dumb baseline, or say you didn't

Every task carries cheap baselines, majority class, sequencing depth alone, PCA plus logistic regression, nearest centroid. The reported quantity is the delta, not the raw score, and a card that fails to beat its strongest baseline says so in its own caveats. ResultCard refuses to validate with an empty baseline set.

Why: attention-derived edge scores were shown to add no incremental value over gene-level features for perturbation-target prediction (BMC Genomics).

Report the ceiling

A score is uninterpretable without what the data actually yields. Supervised tasks carry an empirical ceiling swept over standardisation, dimensionality and regularisation; reference-scored tasks carry the coverage bound imposed by an incomplete reference. Controls refuses to validate without a ceiling or an explicit stated reason none applies.

Why: partial reference labels cap apparent performance regardless of model quality (BioSystems).

Report reversal risk, not a bare rank

An ordering of models is a claim that usually does not survive resampling. Orderings ship with the probability that each pair flips, or, where those replicates are not retained, they ship explicitly marked as point estimates rather than rankings.

Why: rankings reverse across evaluation-protocol axes at rates that make bare leaderboards misleading (arXiv:2603.03493).

Stratify by the confound that actually bites

For single-cell work that confound is the donor, not the cell. Splits are donor-disjoint by default, and group_disjoint_split raises rather than quietly falling back to a leaky split when a dataset cannot support one. Bootstrap resamples donors, not cells, because cells within a donor are not independent and cell-level resampling understates uncertainty.

Why: aging signal that looked robust did not replicate under donor- and composition-aware controls (Biogerontology).

Kill it with a null first

Labels are permuted within strata matched on a technical covariate, sequencing depth, and the metric recomputed. Signal explainable by depth alone does not survive. The null result, its p-value and the covariate it matched on are stored on the card.

Why: raw perturbation shifts in human cortex proved heavily expression-confounded, with no gene surviving expression-matched nulls (Computational Biology and Chemistry).

Pin everything

Dataset hash, checkpoint source, code commit, seed and readout budget in every card. Preprocessing is part of the experiment: the processed matrix is hashed together with the parameters that produced it, including which matrix in the source file was taken to be raw counts.

Why: two groups reporting on "Tabula Sapiens immune" are not comparing the same thing unless every one of those choices matches.

The Standard is enforced in sonde/controls/ and sonde/report/card.py. See the results and the registry.