The Systems Thinker on data as theory residue

Concept map. Nodes: the world W(t), the schema S, the rows R, the query workload Q(t) (the questions people actually run against the database), and the designer D.

• W → R, gated by S: only world-states that fit S become rows.
• Q → R: being read is what keeps rows live.
• W → Q: the world decides what gets asked.
• A mismatch between S, Q and W pushes D to change S. That change is a migration.

This forms one negative-feedback loop with a very long delay. sisuon draws the boundary around S and R. I’d extend it to include Q (see A3).

A1. “A database doesn’t store data.” Formalized: S defines a region of tuple space, and each row is one point inside it. This partially holds. No row is free of theory; it is the idea that observation depends on theory, applied to storage. But S only fixes which space. The rows still say which point, and that remainder is genuinely empirical. The precise version: the database stores data in the theory’s coordinates, and loses whatever falls between the axes.

A2. Schema as niche; constraints as selection pressure. The niche mapping holds well. A schema is close to a Hutchinsonian niche: a many-dimensional region where a record can persist. Remove a constraint and the region changes, so “a different niche” is strictly true. “Selection pressure” doesn’t fully hold. Selection needs variation and different survival rates over time. NOT NULL is a hard yes/no test at insertion. It works like a membrane, not a gradient. But a real selective loop exists one level up. Rejected inserts force the people and processes upstream to produce world-states that conform. Bowker and Star documented this for classification systems. The pressure acts on the processes that generate world-states, not on rows. That is where “against any world-state” correctly points.

A3. Fit decays; the fossil. Rows don’t change, S stays fixed, and W(t) drifts. Fit is the distance between them. This holds, with one joint. The text gives two tests for a fossil: “no longer fits any live query” (usage) and “a niche that no longer exists” (mismatch with the world). These come apart. A row can fit the world and still go unqueried, or misfit it and still feed a quarterly report. Only the usage test can be measured, and it implies that the real environment is Q(t). The questions are the ecology.

A4. Migration as extinction. This breaks at the decisive point, and the text records the break itself: orphans “don’t disappear… They’re kept.” Extinction without death isn’t extinction. A better mapping is a designed niche shift, closer to domestication, that turns records into fossils without removing them. Migrations also transform the records that survive (backfills, type casts), which extinction never does.

The duality claim holds precisely, as exaptation (Gould and Vrba): a structure shaped for one function gets reused for another. A column that answers two questions is already adapted to a niche that doesn’t exist yet. The full argument is in dust-selects-for-the-dual. This text relies on it rather than establishing it.

A5. Technical debt as fossil niche. Formalized: structure X persists if and only if C_remove(X) > C_tolerate(X), where C is cost. The inequality is right, but it describes a vestigial structure, kept inside a living system, not a fossil. And “evidence that the original theory was too singular” fails as a general claim. Much debt is a deliberate shortcut that never had a theory behind it.

Cross-references. decay-as-inventory-ritual-as-threshold closes a loop this text leaves open. Orphaned records, “inventory without function,” are exactly the unattached variation a later migration can reuse. The cycle runs: decay → inventory → exaptation → renewed fit. api-as-fossil-with-handshake claims the “same structure.” Its framing of form as “the invariant across transition” suggests the schema is what persists, homeostatically, across migrations. The excerpt is too short to test this further.

Assessment. The strongest claim is the shortest: “Every index is a bet on which questions will matter.” An index is a structure tuned to the query distribution q expected at design time. When the real workload drifts to p(t), the expected extra cost rises. Coding theory has a close analogue: a code built for the wrong distribution pays a penalty of D_KL(p‖q), the KL divergence between the two. To make the essay precise, define fit as the divergence between the design-time workload and the current one, measured from query logs. Then “decay of fit” stops being a figure of speech and becomes a curve you can plot.