fetch-series knowledge base¶
How to get from one public sequencing accession to another, which routes can be trusted for that, and where the archives lose data silently.
Start here¶
- What is this identifier, and who issued it? → Accession types
- Resolving an accession and want to know which route to use? → The route matrix
- Want to see every route at once, with what each one resolved? → The route graph
- A series came back empty and you suspect it shouldn't have? → Pathologies
- Want to report something to an archive? → Upstream reports
- Need sample-to-run links across archives? → The relation table
- Which files should I actually download? → The file layer
- Is this even 10x? → The assay layer
- What tissue, disease, donor is this sample? → The metadata layer
- Replacing
fetch10xmeta? → The parity record - Want to know how a route gets chosen at runtime? → The resolver
- Wondering how any of this was measured? → Benchmarks
- Want to know why a route is ranked where it is? → Decisions
The rules¶
| Rule | Why | |
|---|---|---|
| 1 | Evidence before defaults. No route ranking ships without a survey behind it. | Every plausible-sounding preference measured so far has been wrong at least once. |
| 2 | Empty is not failure. "The archive holds no such link" and "I could not find out" are different answers. | Conflating them made BioProject → GEO look broken when most of its empties are correct. |
| 3 | A disagreement is the evidence. Agreement between routes is cheap and tells you little. | The 6 series where two GEO routes disagree taught more than the 144 where they agreed. |
| 4 | Cite the accession. Every claim names the accession that motivated it. | "Sometimes GEO omits the relation" is not actionable; GSE135325 is. |
| 5 | Coverage is not correctness. A route that answers more often can still answer worse. | gds_summary beats soft_family on every headline number and sends 190 series to a project holding 7,591 unrelated runs. |
| 6 | A confident identifier is worse than none. A value a search index will trust must be asserted only where an exact match supports it. | disease = ASD exact-matches MONDO's synonym for atrial septal defect, in an autism study, with exactly one hit and nothing ambiguous about the lookup. |
Each rule is a decision with evidence behind it, written up in Decisions — including what would reverse it. A decision with no stated failure condition is a preference, and preferences do not ship here.
What has been measured¶
BioProject → runs, over 12,755 BioProjects, counted at the unique-run level:
| Route | Unique runs | Failed | Surveyed |
|---|---|---|---|
ENA portal filereport |
783,936 | 0 | 2026-09-15 |
NCBI efetch runinfo (via ELink) |
786,586 | 0 | 2026-10-26 |
SRA sra-db-be (via ELink) |
785,656 | 0 | 2026-10-26 |
NCBI efetch runinfo (direct [GPRJ]) |
784,976 | 0 | 2026-10-26 |
SRA sra-db-be (direct CGI) |
784,935 | 0 | 2026-10-26 |
All five agree to within 0.4%. Until 2026-10 this table said ENA held 111,665 runs efetch
never returns. That gap was one EFetch request stopping at 10,000
records, and sra-db-be's
was its history form stopping at 5,000 runs.
Worked around, the NCBI routes miss 162 to 1,270 of ENA's runs and hold 2,261 to 2,812 it
does not, six weeks on from its census. GEO series → experiment is covered under Routes.
The five findings that changed what this tool does¶
| Finding | Where | |
|---|---|---|
| 39.2% | of runs have an ENA fastq missing the read that carries the cell barcode — downloads cleanly, checksums correctly, and no read can be assigned to a cell | pathology |
| 17.6% | of GEO series with SRA data report none through NCBI's elink gds→sra, with zero errors |
pathology |
| 125,271 | BioSamples ENA finds that NCBI's link table does not, 93.6% of them in EBI-registered projects | route |
every SAMEA/SAMD |
BioSample that efetch is asked for by accession comes back as a different sample — prefix stripped, remainder read as a UID, HTTP 200 |
pathology |
| 41% | of the BioProjects esummary db=gds recovers for series GEO leaves empty are species-level umbrellas |
route |
| 0.77% | of runs that were reprocessed successfully no longer exist in either archive | route |
Four of the five contradicted what this project's own code assumed before anyone counted.