← writing  /  experiments

An Ontology Is 12,284 Yes/No Questions

Mapping the entire Disease Ontology onto twenty PubMed abstracts by asking one question per term — a quarter-million judgments, in about five minutes, for roughly a dollar.

Jung Hoon Son, M.D. · September 2026 · experiment notes · python, Jev, Disease Ontology (CC0), NCBI E-utilities

01 — the ideaGrounding as parallel Q&A

Tagging a document with ontology terms usually means a pipeline: dictionary matching, entity linkers, maybe an embedding index. There is a dumber approach that turns out to be practical. Take every term in the ontology and ask, for each document, a single yes/no question — “does this document mention or clearly imply this concept?” — and let a model answer all of them.

The tool is Jev, TypeSafe’s System One decision model: each question is a noul and comes back with a probability, and ~100 questions ride in a single API call, evaluated in parallel. So “map the Disease Ontology onto this abstract” stops being an infrastructure project and becomes 12,284 nouls batched a hundred at a time.

02 — the runThe whole ontology, twenty papers

Twenty PubMed abstracts on rare genetic disease and clinical phenotyping, fetched with NCBI E-utilities. Against them, the full Disease Ontology — 12,284 terms — at batch size 100, four calls in flight at a time:

245,680
judgments
2,460
API calls
5m 14s
wall time
0.35s
median call
23.0M
input tokens
~$1
total cost

Eleven retries out of 2,460 calls, p95 call latency 0.74s. Every one of the twenty abstracts came back with at least a handful of matches at probability ≥ 0.5 — 393 matches in all.

03 — what it foundMatches per abstract

Fig 1. Disease Ontology terms matched at noul probability ≥ 0.5, per abstract. Bars are labeled with the top-scoring specific term for each paper.

The counts track how phenotypically dense the abstract is — a 251-patient genotype–phenotype series lights up more of the ontology than a methods paper — and the top hit is usually the paper’s actual subject. Cystinuria gets cystinuria (0.99). The primary ciliary dyskinesia review gets primary ciliary dyskinesia. Every abstract also picks up the boring ancestors — genetic disease, disease — which is correct, and which a post-order tree walk would let you collapse.

04 — where the time goesBatch size is the only knob that matters

A separate scaling run varied how many questions share one call. Call latency is flat from batch 5 to batch 100 — the fixed cost of the request dominates — so the per-judgment marginal cost keeps falling until the prompt itself gets heavy:

Fig 2. Median latency per API call (bars) and the implied milliseconds per judgment (labels) at each batch size, 3 repetitions per size. Around batch 100 each judgment costs ~2.6 ms of wall time.
Caveat: this measures speed and cost, not correctness. The matches look right on inspection, but there is no gold-standard annotation here — precision/recall against a curated corpus is the obvious next experiment.

05 — why I find this interesting

Ontology grounding has always been the unglamorous prerequisite of clinical data work — the thing you need before cohort definitions, phenotype risk scores, or any FAIR-adjacent plumbing. The historical barrier was never the ontology; it was the linker pipeline. If a quarter-million term-level judgments cost a dollar and five minutes, the barrier is mostly gone, and the interesting questions move up a level: how do you calibrate the probability cutoff, how do you aggregate term hits up the is-a hierarchy, and what does a document × ontology score matrix let you build when it’s this cheap to recompute?

Method: each of 12,284 Disease Ontology terms was posed as one yes/no noul per document (“does this document mention or clearly imply this concept?”), batched 100 per call via Jev, TypeSafe’s System One decision model. Documents: 20 PubMed abstracts (“rare genetic disease clinical phenotype”) via NCBI E-utilities. Ontology: Disease Ontology (CC0). A match is noul probability ≥ 0.5. Companion tutorial: Intro to Biomedical Ontologies (Owlready2).