Skip to content

fetch-series knowledge base

How to get from one public sequencing accession to another, which routes can be trusted for that, and where the archives lose data silently.

Start here

The rules

Rule Why
1 Evidence before defaults. No route ranking ships without a survey behind it. Every plausible-sounding preference measured so far has been wrong at least once.
2 Empty is not failure. "The archive holds no such link" and "I could not find out" are different answers. Conflating them made BioProject → GEO look broken when most of its empties are correct.
3 A disagreement is the evidence. Agreement between routes is cheap and tells you little. The 6 series where two GEO routes disagree taught more than the 144 where they agreed.
4 Cite the accession. Every claim names the accession that motivated it. "Sometimes GEO omits the relation" is not actionable; GSE135325 is.
5 Coverage is not correctness. A route that answers more often can still answer worse. gds_summary beats soft_family on every headline number and sends 190 series to a project holding 7,591 unrelated runs.
6 A confident identifier is worse than none. A value a search index will trust must be asserted only where an exact match supports it. disease = ASD exact-matches MONDO's synonym for atrial septal defect, in an autism study, with exactly one hit and nothing ambiguous about the lookup.

Each rule is a decision with evidence behind it, written up in Decisions — including what would reverse it. A decision with no stated failure condition is a preference, and preferences do not ship here.

What has been measured

BioProject → runs, over 12,755 BioProjects, counted at the unique-run level:

Route Unique runs Failed Surveyed
ENA portal filereport 783,936 0 2026-09-15
NCBI efetch runinfo (via ELink) 786,586 0 2026-10-26
SRA sra-db-be (via ELink) 785,656 0 2026-10-26
NCBI efetch runinfo (direct [GPRJ]) 784,976 0 2026-10-26
SRA sra-db-be (direct CGI) 784,935 0 2026-10-26

All five agree to within 0.4%. Until 2026-10 this table said ENA held 111,665 runs efetch never returns. That gap was one EFetch request stopping at 10,000 records, and sra-db-be's was its history form stopping at 5,000 runs. Worked around, the NCBI routes miss 162 to 1,270 of ENA's runs and hold 2,261 to 2,812 it does not, six weeks on from its census. GEO series → experiment is covered under Routes.

The five findings that changed what this tool does

Finding Where
39.2% of runs have an ENA fastq missing the read that carries the cell barcode — downloads cleanly, checksums correctly, and no read can be assigned to a cell pathology
17.6% of GEO series with SRA data report none through NCBI's elink gds→sra, with zero errors pathology
125,271 BioSamples ENA finds that NCBI's link table does not, 93.6% of them in EBI-registered projects route
every SAMEA/SAMD BioSample that efetch is asked for by accession comes back as a different sample — prefix stripped, remainder read as a UID, HTTP 200 pathology
41% of the BioProjects esummary db=gds recovers for series GEO leaves empty are species-level umbrellas route
0.77% of runs that were reprocessed successfully no longer exist in either archive route

Four of the five contradicted what this project's own code assumed before anyone counted.