CU23 — Workers and jobs, live

from somatize.record import fleet, machines

fleet(store, run="tuesday")        # what each machine did, and what it says it is
machines(store, run="tuesday")     # drawn: working against waited on

The last row of the observability plan CU19 opened, and the first thing to decide about it was what not to build.

There is no registry, and there was never a place for one

A machine does work here in exactly two ways, and each already answers is it alive without anybody keeping a list:

	who knows	already there
a worker serving slices (`.at("worker1")`)	the client, which is talking to it now	`left` with the whole round trip, and a `host` on every fact from over there
a machine claiming trials from a folder	the store	`in_flight`, `abandoned`, and is it still writing measured writer against writer

There is no third case. The original keeps a coordinator with a WorkerStatus, a last_heartbeat and a thirty-second timeout — three missed beats at ten seconds — and it needs one because it has a coordinator. CU15 removed that on purpose, and CU18 answered liveness the other way: not does it answer but is it still writing, against the newest write and never against a local clock.

The record, turned the other way up

What was missing is not state, it is a view. The record is written run → forward → node and where is an attribute, so nobody could ask what a machine is doing. fleet inverts it at the price nodes already costs.

The column that earns it is waiting_us: the round trip minus what actually ran over there — the wire, the queue and the codec. Neither half of that subtraction belongs to a node, so no per-node view can produce it, and it is the answer to was sending it worth it. Against a real pair of workers it says the thing immediately: sixty microseconds of work behind 1.2 seconds of round trip is a slice that should have stayed here.

And the half no record can derive

How loaded a machine is, how much memory is left, how long it has been up. Nobody on this end can work that out. So the worker says it, and three decisions fall out of rules that were already written:

It is a level of its own. A load average is not a fact about a graph, and a variant for it in the core’s Fact would be the engine learning what a machine is — the mistake that keeps loss out of the core. So the vocabulary lives in transport/, where a host is already a thing.

It crosses flat. Fact::Said { kind, pairs } is a carrier and not a vocabulary: what the core learns is that other levels exist and one of them may be speaking from another machine. (kind, pairs) is the shape CU20 named as where the levels meet, and it is what flattened already produces.

Which turned out to cost nothing on the wire. Answer::Saw already carries a Fact, the client already relays one straight to its watcher, and the engine already wraps whatever comes back in Elsewhere — so a reading arrives saying which host it came from without one line attributing it. No message was added to the protocol and no trait grew a method.

It is read and never judged. No bound anywhere near it. Whether 0.9 busy is trouble is an opinion at a threshold, and this library keeps those in health/ where they can be argued with against a record that has already been written.

The reading is taken before the slice and not after: one taken when the work is over is a reading of a machine that has just stopped, and the question is what it was like while it was asked.

The idle machine, which is the one you came to see

A worker only speaks down a wire when somebody gives it work, so the machine sitting there doing nothing would not be in the picture at all — and that is the one a fleet view exists for. python -m somatize.worker --store DIR --reporting SECONDS is the clock.

It goes to the store and not down the connection, and the pipe was already decided rather than open: CU20’s rule is where a connection is open, facts come back down it; where there is none, they go to the store, and an idle worker’s connection is one nobody is reading. That is measured against the code rather than preferred — the client only reads the socket inside say, so a worker beating while idle writes into a buffer nobody drains: it blocks on the write, stops being able to accept the next job, and what the client eventually reads is the oldest beats. Which is the worst available answer to is it alive now.

One name per machine, rewritten, and not one object per reading. That is CU18’s shape and it buys two things: a store that does not grow while a worker sits there, and liveness for free — the store stamps every write, so quiet_s is a scan with no fetches, measured writer against writer and never against a local clock.

And the name it files under is what the machine calls itself, a hostname and a process, because w1 is the graph’s word and a worker does not know it. Which is a fact about this design and not a wrinkle: the two names only ever meet on a reading that came down a wire, where the client attributed it, and fleet joins them there. A machine that wrote and was never asked for anything is in the fleet under its own name with nothing sent to it — which is exactly what it is.

examples/09-a-fleet.ipynb is the picture of it: three workers, two runs, and the bars flip. A cheap slice is 9.4 ms of which almost all is waiting; the same shape with a slice worth sending is 810.7 ms of which almost all is work. Nothing about the figure changed between them.

What is not in it

This machine. here says nothing about itself. It is the one you can look at with top, and inventing a row nobody had to send is not worth the line.

Questionnaire

The fleet (soma-python/tests/test_record.py)

The reading (soma-fabric/wire/tests/unit/machine.rs)

a reading crosses as a flat fact and not as a variant of its own
what nobody measured is absent and not zero
a reading of this machine says how long it has been up wherever it runs
nothing in it is a judgement