Skip to content

CU29 — A kept value says where it came from

g.forward(x, store=where, stamping={"run": "an-investigation/3847d0c1"})
$ cat where/names/*/* | jq -r .meta
[["node","embed"],["fingerprint","920aac16"],
["input","sha256:1064…"],["env","2ffccc9569ca"],["run","an-investigation/3847d0c1"]]

Status: closed. Executor::stamping and the INPUT constant in soma-core/src/execution.rs with 4 tests, somatize/_environment.py, and test_provenance.py (11).

The question: what is a hash six months later?

Section titled “The question: what is a hash six months later?”

A store outlives every process that ever wrote to it. What survives in it is a pile of names that are hashes of recipes, and a recipe does not run backwards: from a key there is no path back to what made it. Inside one afternoon of trying five things, that becomes five sets of intermediates nobody can tell apart. They are not wrong. They are mute, which with time is worse.

So what cannot be recovered is written down at the moment it is known.

written bywhatrecoverable later?
the enginethe node, the fingerprint of its codeit already was
the enginethe input, by the name its content hasnever — only a keeper can hash a value
somatizethe environmentnever — it is in no key
the callera run, a commit, an investigationnot by anybody else

Why the environment, when there is already a fingerprint

Section titled “Why the environment, when there is already a fingerprint”

Because a fingerprint stops at what is installed: a distribution goes in by name and version, and the standard library by its bare name, since the interpreter is compared at the greeting rather than hashed into every class. That is right for naming and wrong for provenance — two interpreters name the same node identically, and only one of them produced what is on disk.

It is filed once as env/<digest> and carried on each value as twelve characters. Both, because whoever reads this store back in a year needs the short name to group by and the long one to understand.

Why the core is told nothing about any of it

Section titled “Why the core is told nothing about any of it”

A commit, an investigation, an environment are facts about the world outside a graph. A core with a field for one of them is a core that has learnt a word belonging to whoever stands above it — so stamping is opaque text it passes through untouched, the same division of labour as Meta itself and as the name a study is filed under. The Python layer fills in the environment because that is where soma meets an interpreter; everything else arrives from outside.

Nobody has to remember. Four of the five are written with no argument passed, and that is the point: provenance that has to be asked for is missing from exactly the runs nobody thought were going to matter.

node, fingerprint and input are refused where somebody is typing, and dropped in the core if they arrive anyway. Not ordering — dropped: whether the first or the last of two pairs wins is the reader’s convention, and the obvious way to read a list of pairs takes the last. A value that came back naming another node would be the one mistake the whole mechanism exists to prevent.

  • A .mapped() node is named by the content of its items, so each item’s key is its own. They are stamped like everything else and attributable one by one; what nothing can do is foresee them, so a reader working from a probe alone will not find them.
  • A slice on a worker says nothing about the graph’s input, deliberately: what arrives at a slice is not what arrived at the graph. Whoever coordinates knows the real one and can send it in a stamp; nothing does yet.
  • Nothing deletes. The Store trait has no forget, and a content-addressed store where two versions legitimately share a blob needs unbinding and a sweep over every name, not an rm. It arrives with a consumer and a decision, not before.