somatize.record

Generated from the package

Rendered from the docstrings of somatize 1.0.1 itself, so every signature and every paragraph below is the one the installed package carries. Nothing on this page is written twice.

What happened, read back.

A Recorder writes one record per forward; this is how it is read:

from somatize import Recorder, Store
from somatize.record import curve, forwards, nodes, runs

store = Store("/scratch/runs")
Trainer(g, objective=..., optimizer=...,
        watching=Recorder(store, run="tuesday", summarising=["loss"]))

runs(store)                            # what is in here at all
forwards(store, run="tuesday")         # step by step, one scan
curve(store, run="tuesday")            # the losses, one scan
nodes(store, run="tuesday", last=50)   # who spent the time, a fetch each

Functions and not a type, like gather and take: what is being read is a folder, and a class around a store would only be the store with a longer name.

There are two ways to see a run and they are not rivals. While it is going, what you want arrives at watching= and costs nothing; when it is over — or when it is another machine’s — there is no connection and a scan is all there is. Both answer in the same shape, so whatever draws one draws the other.

Classes

Live

Live(*, title: str = 'live', smooth: int | None = None, every: int = 1)

A run drawn while it happens. Hand it to watching=:

live = Live()
live                                   # the cell shows it
g.forward(x, watching=live)

With ipywidgets it redraws in place; without one it still tallies everything and figure() gives the same picture. It holds one row of numbers per forward, so watching for an afternoon does not grow with it, and keeps nothing on disk — a Live beside a Recorder is the normal pairing.

Live.figure

Live.figure() -> Figure

The same figure progress draws, from what has arrived so far.

Live.widget

Live.widget() -> Any

A FigureWidget that redraws in place, or None without ipywidgets.

Asked for by hand rather than made on construction: building one imports ipywidgets, and a Live used from a script has no business needing it.

Functions

curve

curve(store: Store, *, run: str, of: str = 'loss.value') -> list[tuple[int, float]]

One number per forward, as (forward, value) pairs — a training curve. of names the field: loss.value when the recorder summarised loss, and otherwise anything a fact carries. Free when the field is in the record and one fetch per forward when it is not, which curve_costs says out loud.

curve_costs

curve_costs(store: Store, *, run: str, of: str = 'loss.value') -> str

Whether curve would scan or fetch: "scan" or "fetch".

Asked before drawing something ten thousand steps long, and the answer is "fetch" when the recorder was not told to summarise that kind.

facts

facts(store: Store, *, run: str, forward: int) -> list[Fact] | None

Everything that happened in one forward, in the order it arrived. The detail, and the one call that costs a fetch. Each fact is the same dict a watching= callable was handed, so what you looked at live is what you read back. None for no such forward, which is not a failure.

fleet

fleet(store: Store, *, run: str, last: int | None = None) -> list[Row]

What each machine did, as the inverse of nodes. The record is written run → forward → node with where as an attribute, and this turns it the other way up.

There is no registry and no heartbeat. A machine is here because it did something, and what it did is already written down.

busy, memory, cores, up_us and served are the half no record can derive — the worker says them itself, down the connection that is already open. The newest reading and not an average; None is a machine that did not say.

waiting_us is the column that only exists up here: the round trip minus what actually ran over there — the wire, the queue and the codec — and neither half of that subtraction belongs to a node.

A scan and a fetch per forward, the same as nodes; last=N reads only the last N.

gantt

gantt(store: Store, *, run: str, forward: int = 0) -> Figure

One forward on a timeline: what ran when, and what was waiting.

The picture spent cannot draw. A total says a node cost four hundred milliseconds; this says whether those were beside the rest of the graph or in front of it.

Every fact carries began_us, so a Wave shows as overlapping bars. A slice that ran elsewhere counts from its own start and is shifted by the left it arrived under — two wall clocks would not have composed. One forward and not an average, because an average of timelines is not one.

machines

machines(store: Store, *, run: str, last: int | None = None) -> Figure

The run, per machine: what each one worked and what it was waited on.

The inverse of spent, and the split is the whole of it: a bar is the round trip in two parts — time spent working, and time waited on, which is the wire, the queue and the codec. Neither half belongs to a node, which is why no per-node view can draw this. Costs what fleet costs.

forwards

forwards(store: Store, *, run: str) -> list[Row]

Every forward of that run, in order, as the record says it. One scan and no fetches, which is what makes this what a progress view reads in a loop. Anything the recorder was told to summarise comes back under its own <kind>.<field> name, as text.

nodes

nodes(store: Store, *, run: str, last: int | None = None) -> list[Row]

What each node did over the run, added up: how many times it ran, how long in total and on average, how often it was read back instead, and where. It costs a fetch per forward, because which node did what is in the blobs; last=N reads only the last N.

progress

progress(store: Store, *, run: str, smooth: int | None = None) -> Figure

How a run went, step by step: the loss, the time, and what broke. One scan when the recorder summarised the loss and a fetch per forward when it did not — curve_costs says which. smooth is the window of the rolling mean, in forwards; 0 draws only what was measured.

runs

runs(store: Store) -> list[Row]

Every run this store holds, newest last. One scan and no fetches: how many forwards each has, how many broke, when the first and last were written, and how long they took. The one to call first, because a store holds whatever anybody put in it.

spent

spent(store: Store, *, run: str, last: int | None = None) -> Figure

Where the time went, added up per node. Bars are coloured by where the node ran, from the same table the graph is drawn with. A fetch per forward, because which node did what is in the blobs; last=N reads only the last N.

standing

standing(store: Store) -> dict[str, Row]

Every machine writing readings into this store, as {id: reading}.

The idle half: a worker says what it looks like on a clock whether or not anybody is asking it for anything, and it goes here rather than down a wire because a client only reads the socket while a job is in flight.

Keyed by what the machine calls itself, since on this path there is no client and w1 is the client’s word; fleet joins the two. quiet_s is how far behind the newest reading each one is, measured writer against writer and never against this machine’s clock. One scan and no fetches.