Skip to content

The crates

The workspace is nine crates. Eight are published to crates.io as somatize-*; the ninth is the extension module and carries publish = false, because its API is Python’s.

The directory is named after what it holds and the package after what it is called on crates.io — soma-core/ builds somatize-core. The names diverge because soma and soma-core on crates.io belong to other people, and somatize was already ours.

cratedirectorywhat it is for
somatize-coresoma-core/the graph, the plan of how to run it, and the engine that walks that plan
somatize-storesoma-store/where a computed value is kept: bytes by content, and names that point at them
somatize-datasoma-data/where the data comes from, and what it is once it arrives
somatize-studysoma-study/a search over configurations, and what each trial did
somatize-healthsoma-health/whether what a run did is healthy — an opinion, and it says so
somatize-treesoma-tree/what an edit did to a graph, said before anybody runs it
somatize-fabric-wiresoma-fabric/wire/carrying a slice of a plan to another process, and bringing back what it produced
somatize-fabric-brokersoma-fabric/broker/the name a graph gave a host, turned into a way of reaching it
_somatizesoma-python/the PyO3 module. Not published as a crate

The column that decides most of the layout. Counted with cargo tree --edges normal, default features, the crate itself included:

cratecrates pulled in
somatize-core1
somatize-study1
somatize-health1
somatize-store24
somatize-fabric-wire25
somatize-fabric-broker26
somatize-tree55
somatize-data64
_somatize152

Three of them are one, which is to say they have no dependencies at all.

For the core that is a position: it provides five holes and fills none of them, so there is nothing for it to depend on. serde is behind a feature and off by default.

For study/ and health/ it is stronger than a position, it is what makes an invariant testable. health/ is numbers in and flags out — no measuring, no clock, and not even the core. That is what makes a diagnosis has to be reproducible from the stored record, without training again a test rather than an aspiration: change a bound and ask again, and the record has not moved.

Why fabric is two crates and not one with modules

Section titled “Why fabric is two crates and not one with modules”

The dependency between them runs one way only: broker depends on wire, because resolving a name has to end in something that can carry a slice. Nothing in wire reaches for a broker — grep it and there are no hits. That asymmetry is what makes the cable knows nothing about the rendezvous a fact the compiler checks rather than a sentence in a document, and it is why a worker needs no broker at all in order to serve a slice: it is being talked to, and finding out where it is was somebody else’s problem.

There is no tokio anywhere in this list, and Store is synchronous on purpose. That is the reason SQL is not in the data layer: every driver worth using carries a runtime, and an async Store would be async in every caller of it.

data/ takes Arrow as the type and leaves the tool to whoever wants one — an expression engine was measured at around 370 crates when that decision was taken, and the worker that only tokenizes has no use for expressions. Sixty-four is what Arrow and parquet cost on their own.

cratefeaturedefaultwhat it adds
somatize-coreserdeoffthe plan and the keys, serialisable
somatize-stores3offStore.on_bucket(...), the second implementor
somatize-treeclionthe somatize-tree binary, and clap with it
_somatizeremoteonthe wire and the broker

Cargo features do not reach the wheel. pip install somatize[viz] adds dependencies of Python; the Rust is compiled in. What the features are for is the worker’s image, which is built --no-default-features, and not for anyone installing the package.

The original’s workspace is thirteen crates. Three of those names survive — core, store and the Python module — and ten have no counterpart at all: the soma facade, plus -compiler, -runtime, -macros, -worker, -memory, -agent, -llm, -mcp and -coordinator.

Nothing had to be done with them. They simply stopped publishing versions, and versions on crates.io are immutable, so 0.5.1 still resolves for anyone who pinned it.

-compiler and -worker are the two worth a sentence, because their absence is a decision and not an omission. Compiling a graph is compile, a free function in somatize-core; and a worker is a binary in the Python package, python -m somatize.worker, because what it needs is an interpreter and not a crate.

For what is inside each one, see the Rust API reference.