Skip to content

When metadata may break a tie

Status: decided 2026-10-23 and permitted by the rule set before measuring. Built 2026-10-26: fetch inspect run offers mapping_from_metadata. Evidence: metadata against the reads and the pipeline

The question

fetch inspect cannot always tell 3′ v2 from 5′ v1/v2.

  • What the two share. The same whitelist, a 16-base barcode and a 10-base UMI.
  • What tells them apart. The strand of the cDNA read, and the template-switch oligo after the UMI.
  • Where that fails. A 26-base barcode read cannot show the TSO, and the strand signals do not always agree. Under classifier 0.4.0, 94 of 979 samples read are left ambiguous there, and get no mapping block.

A protocol that names "Chromium Next GEM Single Cell 5′" would settle such a sample before anything is downloaded. The question is whether a block may rest on it.

The rule, set before the numbers

The round's plan, approved on 2026-10-23, said: if the metadata's 3′/5′ call matches the pipeline's strand with zero errors on the design half, a block for an ambiguous read call may use it. It names the metadata as its source, and it stays unvalidated until scored on the held-out half.

What was measured

Design Held-out
metadata kit's strand against the pipeline's, at scale 340 of 340 606 of 608
reads ambiguous 3′ v2 / 5′, metadata names one of the two 7 2
…settled as the pipeline mapped it 7 of 7 2 of 2
metadata against the reads, where both name a kit and disagree the pipeline sides with the reads, 20 of 20

The design half meets the rule. The held-out half then found the failure mode, though not on a tied sample. GSE121380's protocol names a 5′ kit for all thirteen samples. The pipeline mapped two of them as 3′, and the reads call one of those 3′ v2 at HIGH. Had the reads abstained on those two samples, the metadata would have broken the tie wrongly.

The decision

Metadata may break the 3′ v2 / 5′ v1/v2 tie, and only that tie, on these terms:

  1. Only where the reads abstain. It never overrides a read call. In all 20 disagreements, the pipeline sided with the reads.
  2. Only where the metadata names exactly one of the two kits. A study that names both says nothing about any one of its samples.
  3. Offered beside the read-derived block, never merged into it. The block's own rule is that soloStrand comes from the strand the reads were measured on, never from a kit's name (inspect/readiness.py). A tie broken by metadata is a different kind of claim, so it is offered separately, as starsolo_paired is:
    • it names its source, metadata, and the fields that matched;
    • it carries the measured agreement above: 9 of 9 ties, and 946 of 948 samples at scale with both errors in one study.
  4. Validated on 9 ties. That is a small number, and it says so.

What it buys is 9 of 94 abstentions. The other 85 have metadata that names neither kit, or both.

How it is built

The harvest. fetch inspect run reads the run's metadata the way the comparison read it (inspect/metadata.py), with two requests at most:

  • ENA's read_run report for the run, with the protocol fields and the sample alias;
  • where the alias is a GSM, GEO's record for that sample, whose free-text fields are the ones the SOFT family gives.

A failure is recorded in the report as why no call was made, redacted, and never stops the read report. --no-metadata skips the harvest.

The block. readiness.from_metadata returns None unless every term above holds:

  • the read call is ambiguous between exactly 3′ v2 and 5′ v1/v2;
  • the metadata names one of the two and not the other;
  • the reads did not measure a strand against that kit;
  • the library is not typed as anything but gene expression;
  • the copy is not unfit.

When it holds, the block is laid out for the named kit. Its soloStrand is the kit's, and the block says so, with:

  • source: metadata;
  • the phrases that matched, under grounds;
  • the measured agreement above.

It sits beside mapping as mapping_from_metadata. mapping itself stays empty, and so does the fit verdict: metadata never makes a copy fit.

Run live. SRR9141217's reads leave it tied, and its protocol names a 5′ kit. It gets --soloStrand Reverse and SC5P-R2 from the metadata, and the pipeline mapped it Reverse.

A model's baseline

The deterministic tiers now have a measured baseline on a labelled set. A tier-3 (LLM) metadata pass would have to beat it.

  • The labelled set. truth.csv: 5,762 samples the pipeline mapped, labelled by the whitelist it chose and the strand its test mapping found. The design half has 2,695 samples and the held-out half 3,067. A model's prompt would be developed on design and scored once on held-out.
  • The baseline to beat:
    • Coverage. The deterministic tiers name exactly one kit for 964 of 5,465 harvested samples (17.6%).
    • List error. The kit implies another list than the pipeline's for 64 of 964 (6.6%).
    • Strand error. It implies the other strand for 2 of 948 (0.2%).
  • What a model must also do. Decline where a study ran two kits, and not turn a study-level protocol into a per-sample claim. Every error measured here is that one mistake.

Building tier 3 is out of this round.

What would change this

  • A tie broken wrongly. A held-out sample where the reads abstain and the metadata names the other kit from the pipeline's would end the exception, not adjust it.
  • A classifier that settles the tie from the reads. It would make the exception unnecessary. The reads won all 20 disagreements, so they remain the instrument to improve.