Skip to content

The route graph

Every link between records, files and documents the archives offer, and what is known about each. Columns are archives — GEO, NCBI, EBI, DDBJ, ArrayExpress — and rows are levels, from datasets down to read files, supplementary files and metadata documents. A measured route is a solid line: its ink is the share of its source's queried accessions its survey resolved, its width how many of them reached this target. A surveyed link is a hollow slate line of one width: a link survey's seeded draw found it usable, and it carries counts and never a rank. An untested link is a thin dotted line with a hollow arrowhead: no survey has measured it, and one live request found it answering — and, into a file or document, found something usable.

How to read it

Measured routes / Measured + surveyed / All known links switches between the declared routes alone, the routes and every link a link survey measured, and every link the catalogue has confirmed. All is the default, because the gaps between the routes are the point of drawing it. Submissions, analyses and GEO platforms are hidden until asked for: they are records about the data, not levels of it, and they cross every row.

Columns, colours and shapes

Each archive's records sit in its own column, and colour repeats the column: GEO orange, NCBI blue, EBI green, DDBJ violet, ArrayExpress amber. Neighbouring columns are told apart by every reader, colour-blind ones included; columns two apart are not always (NCBI blue and DDBJ violet), which is why every label also names its archive — SRA run, ENA run, DRA run — and carries the accession prefix under it. A BioProject or BioSample is one record held by all three INSDC archives, and sits on a row of its own above its level, within their columns. Read files belong to no archive: they are grouped by format, and grey.

Shape says what a node is: a stack of sheets for a dataset, a capsule for a sample, a hexagon for a library, a slanted bar for a run, a plain card for a record about the data, a folded page for a file, and a card with a ruled header for a metadata document.

A measured route

A declared route is drawn once per node pair it covers: BioProject → SRR, → ERR and → DRR are three lines of bioproject->run:ena_filereport. Each is cut from the route's own recorded verdicts, by the archive of each accession and the format of each file — filtered measurement, never new evidence (a route's split is not new evidence). Where several routes answer the same direction they run side by side, in the order the resolver tries them: RouteRegistry.ranked() over the evidence, not a hand-written preference (how routes are ranked).

  • Ink is the share of the source's queried accessions that resolved — SRRs alone, for a line from SRA runs — in five steps from pale (under 25%) to full ink (95% or more).
  • Width is how many of those accessions had a result on this target, on a log scale.
  • Hatched marks the routes still measured only by the 2026-05 scripts, which kept a single boolean per BioProject that lumped empty in with failed: the two BioProject → GEO series routes, since the four BioProject → run routes were re-surveyed on 2026-10-26. Their share cannot be computed, and is drawn as unknown rather than guessed.
  • A line into a read format says who made the files in brackets: submitted, generated by the archive, an SRA object, or mixed.

A pair the survey never queried is not drawn as measured: it is probed like any other link. A pair the survey queried and never reached is a measured absence: listed in the panels and the matrix, with no line. Hover or click a route for its split and the whole route's evidence.

One hollow line per node pair where a link survey found something usable: the probe's request made over a seeded draw of 300 of the source node's accessions, or all of them where there are fewer (link surveys). Clicking it lists each mechanism with, per corpus, how many accessions it found something usable for out of how many were queried, the resolved, empty and failed counts, the distinct values, each check's tally, and the date.

It has one width whatever it found, and no rank, on purpose. A draw of 300 puts up to ±5.7 points on a yield, which says whether a link answers always, sometimes or never and cannot say which of two answers better (a link survey is not a ranking). A link whose survey found nothing usable is not drawn; it is listed in the panels as surveyed, never usable.

One dotted line per node pair, however many mechanisms answer it. Their names appear along the line once you zoom in, and in full on hover. Clicking it lists each mechanism's request, the field the link is read from, whether the archive documents that field, and the probe that confirmed it: which accession, on which day, and what came back.

A line into a read format is drawn only where the file found is valid: it downloads at the size the archive published, the archive publishes its md5 and size, and it is complete for 10x — the barcode read and the cDNA read both there. A supplementary file must download and be size-checkable, a metadata document download and parse. The panel lists each check and what it said.

A probe proves a link exists and measures nothing, so an untested line carries no counts, no rank and no ink (a probe is not evidence). A documented link whose probe found nothing, or found something that failed a check, is not drawn; it is listed in the panels and on the catalogue page.

A line out of a metadata document

Every line leaving a metadata document is the second half of the request that fetched it:

  • The request. GEO series → GEO metadata fetches the SOFT family file.
  • The read. GEO series → GEO sample reads that same file for its samples.
  • The line. GEO metadata → GEO sample is the two joined. It is how the reprocessing pipeline itself gets from a dataset to its samples, through SDRF, IDF and SOFT family files.

How these lines behave:

  • They make no request and are not catalogue links. The catalogue's counts do not change.
  • Their style is the record link's. A line is drawn surveyed where the record link it continues is measured or surveyed, and untested where that link is untested.
  • Their panel names both halves. It shows the mechanism that fetches the document and the one that reads it, with the record link's own evidence.
  • Paths go through documents, so Find a path can pass through one.

Only a mechanism that reads the same document continues it. MINiML, series matrix, SRA XML and DDBJ entry documents are fetched, but nothing declared reads them for records yet, so they have no line out.

Panels, matrix and paths

Clicking a node lists everything it reaches in one hop, with what is known about each, and every node it has no one-hop link to. The matrix under the graph is the same information for every pair at once: filled where a route measures the pair, ringed where a link survey does and no route, outlined where only untested links do, struck through where a route measured an absence or a survey found nothing usable, dashed where a link is documented and not confirmed, blank where nothing links them. Clicking a cell selects that pair.

Paths are node paths of up to four hops. Over measured routes the path finder takes, for each hop, the best-ranked route the filters leave visible, which is what the resolver would try first. In measured + surveyed it also uses surveyed links, where no route answers the hop. With include untested it also offers paths through untested links: each pair's shortest paths and those one hop longer.

The data behind this page is printed by uv run fetch routes graph, which produces it the same way the docs build does. Node positions were searched, not placed by hand, by scripts/route_graph_layout.py, within each node's row and archive column. The same search then bends a bundle that would pass under a node it does not belong to, but only where a bend clears every such node: in the default view that leaves 23 route crossings of that kind and 28 untested ones, against 32 and 94 drawn with one gentle curve for all. What is left is mostly BioProject's and BioSample's lines, which reach almost every node below them. Letting those two sit outside the INSDC columns was tried, and the search put both back where they were. Colours were chosen by running the categorical palette through a validator in both themes: every pair of neighbouring columns separates for colour-blind readers, and the pairs that do not are never next to each other.