CU13 — What is remembered, and what is not computed twice

from somatize.torch import Trainer, freeze, parameters

g = Graph.somatize(Encoder().frozen().cached() >> Head())

# training: the Trainer settles what was declared, and keeps what was declared kept
Trainer(g, objective=cross_entropy, optimizer=Adam(head_only, lr=1e-3),
        store="/scratch/soma").fit(data, epochs=20)

# inference: settle it yourself, and the second run does not touch the encoder
freeze(g)
g.forward(Opaque(x), store="/scratch/soma")

Status: closed. 185 tests in the core, 33 in the store, 73 in the transport and 249 in Python.

The case it exists for is labchain’s (SoftwareX, S2352711026000373): an expensive, settled node — an encoder, an embedding — under a head that changes twenty times in an afternoon. What has to be true is that changing the head does not touch the name of what is underneath it.

The question: where does the hash go?

labchain hangs it inside the data: an XYData object carrying the value and its hashes. It has no choice — it has no engine, the pipeline is the user’s code, and the only place left to put a hash is the datum itself.

Here there is an engine, and walk already carries produced everywhere. So the key goes beside it, in a parallel table with the same life cycle: the same copy into a wave’s branches, the same retention in resume, the same merge from what came back over a wire. In exchange, Value grows no wrapper that every forward would have to unwrap, and the Node contract does not change by one character.

The engine is the only one that sees every edge. Anything that wants to travel along an edge without being a value belongs to the engine, not to the value.

The key: a Merkle hash over the recipe, not over the data

key(root) = H(the input, by its content)     ← the only place data is hashed
key(node) = H(identity, state, salt, the keys of its predecessors)
identity  = the name of the class            ← not the fingerprint of the code

From the root down they are hashes of hashes, and that is the whole point: the key is known before anything runs. Naming what a node will produce does not cost a byte of the data it will produce it from, and changing the classifier does not touch the name of the embeddings under it.

Three things deliberately left out of the key:

The fingerprint of the code. In it, a cosmetic refactor would invalidate half a store in silence. It is written beside the value and compared on a hit, which turns the same event into a line on stderr you can act on. The window it leaves open is narrow and known: two classes of the same name with different bodies share a key, and the fingerprint is what says so out loud.
The device. Where something ran is not what it is. What the key cannot see, the user says with .cached(salt="a100-fp16").
The graph. A key names a node’s output, not the run it happened in: that is what makes two graphs share what they have in common.

The prefix rule, which is one line and has two reasons

A node’s output can be kept if nothing upstream of it can change — itself included.

Freezing the node alone is not enough: freezing layer 3 of 5 does not stop the gradient crossing it towards layers 1 and 2. Two independent arguments land on the same line, and that is why it is a check and not a warning:

what is restored from a store is a leaf. The backward pass stops there and everything above it quietly stops training;
the digest of the state is in the key, so a node that still trains gets a new key every step. It never hits; it only fills the store.

At inference the whole prefix is settled by definition, so all of it is cacheable. The question is asked by cacheable(graph, memory) before the first node runs — the engine never sees a graph, so it cannot be the one to ask.

Declaring and obeying, for the fourth time

Memory is the fifth fact, and like the other four it is inert data:

piece	answers
`Graph`	what exists
`Catalog`	who executes it
`Placement`	where
`Plan`	when
`Memory`	what is remembered

The core defines frozen as “this node’s state does not change while the graph runs” — a statement about cache validity, not about gradients, which it still knows nothing about. somatize.torch.freeze is what makes it true, with requires_grad_(False), exactly as a node and not the core is what moves a tensor to a GPU. And the digest of the weights is paid for there, once, because settling is the moment that makes both halves true at the same time.

The fourth hole, and the first that is a decorator

Keeper joins Node, Driver and Transport: the core provides the hole, whoever knows what goes in it is a library. Here it is doubly true — hashing is sha256 and keeping is a directory, and the core has no dependencies at all.

Driver serves, Transport carries, Keeper keeps.

What fills it is somatize_store::Cache, and in Python there is a second one in front: Packing turns every Opaque into bytes on the way in and back on the way out, so the store sees maps and bytes and never learns Python exists.

Decisions taken

1. The key travels beside produced, not inside Value. See above. It also means a node that declares nothing pays nothing, and that without a keeper the table is not even computed.

2. .cached() is opt-in, and not declaring it does not break the chain. A node with no cache still gets a key and still passes it on. Otherwise declaring the cache node by node would be declaring it for the whole graph.

3. A keeper that fails never kills the run. It is said on stderr and the value is recomputed. A cache is an optimization, and one that can kill a run at hour three is not one. Same criterion the worker already applies to a store it cannot reach.

4. The frontier of Opaque moves rather than disappearing. From “an opaque does not travel” to “an opaque nobody registered a codec for does not travel”, which is the more precise of the two. codec(kind, type, dump=, load=) is the register; somatize.torch fills in the tensor’s on being imported, so a graph that keeps tensors and never imports it keeps nothing and says why on stderr. Importing torch is not enough and is not meant to be: registering it from somatize would mean importing torch for everyone who does not have it. What comes back is a leaf, and there is a test that says so, because it will look like a bug the first time somebody sees it.

5. A tensor comes back on the cpu, and weights_only=True. A store shared between machines that only reads back where it was written is not shared at all; and one that unpickles arbitrary objects is a way in. Whoever receives it moves it, which is what a placed node already does with its input.

6. Values and artifacts live in the same store, under two namespaces. value:<key> and artifact:<kind>:<id>. Two questions — a catalog that is not sent twice, a node that is not run twice — one directory, and what keeps them apart is the name.

7. Declared versus injected, for the fifth time. Memory is declared, like the Placement: it belongs to whoever wrote the graph, and it travels. A Keeper is injected, like the Driver and the Transport: it belongs to whoever runs, and it does not. They were one builder call for a while — “neither is any use alone” — and that was false: a coordinator that keeps nothing itself still has to tell a worker what the nodes are, or the one side that does keep things can name none of them. remembering(&Memory) and keeping(&dyn Keeper), and there is a test where this side keeps nothing at all.

8. Whoever obeys is the only one who can check that obedience happened. The core cannot tell a node with no state to settle — a tokenizer — from one whose weights nobody has hashed yet: both arrive as a state of None. Left alone, that is the one failure a cache must not have — two checkpoints of the same class under one name, the wrong tensor back, no error and no warning. So Python asks it, by the same duck it asks for parameters(), wherever a cache is declared: something with state, settled, and no digest, refuses to run and says to call somatize.torch.freeze(g). There is a test that reproduces the bad hit and one that shows two checkpoints settled at the same digest are one name, because that is what says the digest is what the key believes.

Questionnaire

The core (soma-core/tests/unit/{execution,build,memory}.rs)

what is kept is not computed again, and what is kept under that name is what was produced
a different input is a different name, and the node runs
what is above names what is below: another state upstream, another name
the fingerprint of the code is not part of the name, and what is written beside the value is what produced it
a node that keeps nothing still passes its name on
an Opaque root leaves everything below nameless, and it is not an error
nothing is named without a keeper
cacheable names the cached node and the ancestor that can still change
the names a slice brings are not the names it gives, and both cross a Cargo and come back in an Outcome

The store (soma-store/tests/unit/cache.rs)

the pieces of a recipe cannot run into each other: ["ab","c"] and ["a","bc"] are two names
the same recipe is the same name every time; only a root is named by its content
a batch answers in the order it was asked, holes included
a name nobody kept is a miss and not a failure
a kept value is findable by looking at what is in the store
the same bytes under two names are stored once

The worker (soma-fabric/wire/tests/unit/worker.rs)

what a worker already kept is not run again over there — and the same worker without a keeper runs it every time
the name of what ran over there comes back

Python (soma-python/tests/{test_cache,test_freeze}.py)

What did NOT go in

The grain per item (.mapped()): a node that maps over items, with a key per item instead of per node. It is designed and unwritten — Key is public and there is no Keys yet, on purpose: a variant nobody can construct is worse than a variant that arrives late. (CU16 wrote it, and found that it and micro-batches are not the same question.)

Also out: .overwrite(times=1), which is a policy of the run and lives in the executor, not in what is kept · the queryable index — what do I have, from which run, from when — which is a SQLite derived from the records and throwaway, and making it the truth would mean a single writer over NFS · a strict mode for the fingerprint (.cached(strict=True)) · and S3, which arrives the day there is a MinIO to point at, through OpenDAL and as another configuration rather than another implementation. (It arrived after CU19, and that last prediction was wrong on both halves — see there.)

What reviewing this slice found

Three things, and they went in before it was called closed. Written down because two of them were wrong numbers, not rough edges:

a .frozen() declared and never obeyed gave a false hit: two checkpoints of the same class, the same input, and the second run got the first one’s tensor back. Reproduced with a script before being fixed; it is decision 8;
Memory and Keeper went in one builder call, so a coordinator with no store of its own sent an empty table and a worker that did have one could name nothing. It is decision 7;
cacheable was asked only when a store= was given, so a .cached() in the wrong place stayed silent until somebody added a directory to the call — possibly in production. It is asked wherever a cache is declared.

And two more, found writing the worked example that is now test_pretraining.py — which is the argument for writing one:

Trainer had no store=. step called graph.forward(input) and nothing else, so the one case this whole slice exists for could not go through the Trainer at all;
a class’s fingerprint changed when an unrelated global was defined. _names read code.co_names, which mixes global loads with attribute names: self.model puts model in there, and a module with a global called model had its value hashed into the version of a class that never named it. It is CU12’s code, and it is not cosmetic — with project and --strict a worker refuses to run over a mismatch that does not exist.