somatize.worker

Generated from the package

Rendered from the docstrings of somatize 1.0.1 itself, so every signature and every paragraph below is the one the installed package carries. Nothing on this page is written twice.

The generic worker: pip install somatize and nothing else.

python -m somatize.worker --listen 127.0.0.1:7000 [--store /scratch/soma]

An independent process on whatever machine. It starts empty — it does not know what tokenize is — and waits for someone to connect and send it what to build its catalog from. There is no way of giving it one by hand, on purpose.

artifact kind	what arrives	what the worker supplies
project (default)	names, versions and state	the code, from its clone
pickle	the code and the state	nothing

project is what you want when the worker runs in a clone of the project: tens of bytes per node, no coupling between interpreters, and it checks the version — --strict (default) stops with both versions in front of you and --lucky runs anyway and says so on stderr.

Whoever reaches this port runs code here, as their user. There is no authentication and there is not going to be one — that is srun’s and ssh’s job. Bind to 127.0.0.1 and tunnel, or to a private interface inside a cluster.

It does not solve the environment: cloudpickle moves your objects, not torch. That belongs to whoever stands the worker up, and putting it in here cost the original 420 lines and a hot pip install. It fits in one file:

#!/usr/bin/env -S uv run --script
# /// script
# requires-python = "==3.13.*"
# dependencies = ["somatize[remote]", "torch==2.10.0"]
# ///
from somatize import worker
worker.listen("0.0.0.0:7000", store="/scratch/soma")

stdout is the wire, so at startup sys.stdout is redirected to stderr: a stray print() in one of your nodes would otherwise break the protocol.

Classes

Pickles

Pickles()

Turns pickle artifacts into nodes. Python’s Provision: accepts(client, kind) returns None or how this worker identifies itself, and provide(kind, blob) returns the nodes as a dict.

Pickles.accepts

Pickles.accepts(client: str, kind: str) -> str | None

Pickles.provide

Pickles.provide(kind: str, blob: bytes) -> dict[str, Any]

Project

Project(strict: bool = True)

Opens project artifacts: resolves the classes against this clone. No code comes over the wire, so neither cloudpickle nor matching interpreters are needed.

Project.accepts

Project.accepts(client: str, kind: str) -> str | None

Project.provide

Project.provide(kind: str, blob: bytes) -> dict[str, Any]

Provision

Provision(*args, **kwargs)

How a worker turns an artifact into a catalog. A Protocol because there are three of these here and none inherits from the others. It is the seam wire’s Provision trait is filled from, and a user with a fourth way of packing nodes writes one and passes it to listen.

Provision.accepts

Provision.accepts(client: str, kind: str) -> str | None

None to accept, or how this worker identifies itself so the client can see what it disagrees with.

Provision.provide

Provision.provide(kind: str, blob: bytes) -> dict[str, Any]

The nodes that artifact unpacks to, by id.

Strategies

Strategies(**by_kind: Provision)

Several ways of building the catalog, asked by kind and not tried in a chain to see which sticks.

Strategies.accepts

Strategies.accepts(client: str, kind: str) -> str | None

Strategies.provide

Strategies.provide(kind: str, blob: bytes) -> dict[str, Any]

Functions

listen

listen(addr: str, provision: Provision | None = None, opened: Callable[[str], None] | None = None, store: Store | str | None = None, reporting: float | None = None) -> None

Stands on addr and serves whoever connects. It does not return.

provision says what the implementations are resolved with. stdout is not touched — here the wire is the socket — and opened is called once with the real address, so port 0 can be asked for. store answers two questions that stay two: an artifact it already has is not sent again, and a node whose answer is there is not run again.

runtime

runtime() -> str

How this process identifies itself, so a mismatch is refused on connect instead of surfacing inside a loads. somatize’s version goes here, which is why it does not go into each class’s fingerprint.

serve_provisioned

serve_provisioned(provision: Provision | None = None, store: Store | str | None = None, reporting: float | None = None) -> None

Serves slices with the catalog the client sends, until the client closes. Without an argument it opens both kinds.