Codebase Map

Three documentation sections describe this codebase and they answer different questions:

Architecture — the shape. Layers, flow, responsibilities.
Design — the why. What was chosen, over what, and what would change the answer.
Internals (this section) — the what, with file:line. Every public trait, struct and enum; who implements what; who owns what; what is wrong.

Use this section when you need to find something, when you need to know what implements a trait, or when you are planning a change and want to know what it will touch.

Reading order

If you want to…	Read
Get oriented in one page	This page — the spine and the ten types
Explore instead of read	The Architecture Graph — click a trait to see every implementor, a type to see what owns it
Follow the code as it runs	Call Paths — the five traces as one graph, with the hops they share
Know the vocabulary	Foundation — `soma-core` is the dictionary every other crate speaks
Understand how a graph runs	Execution — especially the traces
Understand how an agent works	Agentic Stack — start with D4
Work on the Python API	Python Bridge
Work on remote execution	Distribution — start with D5
Recognize an idiom you keep seeing	Design Patterns
Plan a refactor	Known Debt
Find one symbol	Symbol Index

Notation

Rust has no classes, so a UML class diagram does not translate directly. These pages use a fixed ASCII notation instead — greppable, diffable, and readable in a terminal. It is used identically on every page.

«trait» Name        an interface (≈ UML interface)
   ▲
   ├── Type         realization      (impl Name for Type)

A ──◆ f: T          composition      owned by value or Box — dies with A
A ──◇ f: Arc<T>     aggregation      shared — may outlive A
A ──▷ B             uses / calls     no ownership
A ──? f: Option<T>  optional

[enum] E {A|B|C}    an enum, variants inline
 (!)                a documented deviation — see the Debt Register
  !                 #[non_exhaustive]

There is deliberately no single diagram of all ~250 types. A diagram of 250 nodes is decoration. Six targeted diagrams show the seams instead, and the tables are the diagram at finer granularity:

	Diagram	Page
D0	The ownership spine	below
D1	The node seam — `Filter` / `Step` / `NodeCatalog`	Execution
D2	The execution pipeline	Execution
D3	The cache and journal stack	Execution
D4	The effect loop	Agentic Stack
D5	What crosses the wire	Distribution
D6	The FFI bridge	Python Bridge

A file:line reference is written as plain inline code — `soma-core/src/filter.rs:120` — never as a link. A GitHub permalink would need a pinned commit, and two hundred of them would rot in one commit.

The workspace at a glance

~70 000 lines of Rust across 13 crates, plus a 7 400-line pure-Python package. Published names are prefixed somatize-; directory names drop the prefix.

Crate	Lines	Traits	Page
`soma-core`	11 590	12	Foundation
`soma-macros`	607	0	Foundation
`soma-compiler`	3 118	1	Execution
`soma-runtime`	17 449	12	Execution
`soma-llm`	3 848	2	Agentic
`soma-agent`	620	0	Agentic
`soma-memory`	3 746	2	Agentic
`soma-mcp`	3 267	0	Agentic
`soma-worker`	5 903	0	Distribution
`soma-coordinator`	949	0	Distribution
`soma-store`	1 285	0	Distribution
`soma-python`	6 720	0	Python Bridge
`soma` (facade)	124	0	Foundation

29 public traits total. Not one of them declares an associated type or a generic parameter, so all but StudyIo and Searchable are object-safe — which is why every backend in the system is swappable at runtime without a generic bound leaking into a signature.

One number to keep in mind before judging any file by its length: 60% of soma-runtime is tests. executor.rs is 2 472 lines of which 1 184 are inline #[cfg(test)]; executors/study.rs is 1 915 of which 1 470 are.

The dependency graph, with the trait seams marked

Acyclic, read top to bottom. The arrows on the right are the traits crossing each boundary — those are the joints the system bends at.

soma-macros                       proc macros; no internal dependencies
    │                             ─── generates ──▷ config_hash, impl Searchable
    ▼
soma-core                         types, traits, serialization
    │   « defines: Filter, Step, CacheStore, DataStore, StateStore,
    │     EffectHandler, ActionCache, BlobStore, EventSink, Tracker,
    │     Searchable, AsAny »
    ├──▷ soma-store               ──▷ impl DataStore  (S3, Zarr)
    │
    ├──▷ soma-compiler            « defines: NodeRegistry »
    │        │
    │        ▼
    │    soma-runtime             « defines: Runner, Transport, ForwardStrategy,
    │        │                      Sampler, Pruner, TrialExecutor, PbtExecutor,
    │        │                      StrategyContext, StrategyExecutor,
    │        │                      GradientAggregator, StateAggregator, StudyIo »
    │        │                     ──▷ impl NodeRegistry for NodeCatalog
    │        │                     ──▷ impl CacheStore ×4, EventSink, Tracker
    │        │
    │        ├──▷ soma-llm        « defines: LlmProvider, Tool »
    │        │                     ──▷ impl Step ×3, impl EffectHandler ×2
    │        │
    │        ├──▷ soma-worker     ──▷ impl Transport, impl Filter
    │        │        │
    │        │        ▼
    │        │    soma-coordinator   (reuses soma-worker's wire vocabulary)
    │        │
    │        ├──▷ soma-agent      ──▷ impl Step         ┐ both also depend
    │        └──▷ soma-memory     « defines: KnowledgeBase, Embedder »
    │                 │                                 ┘ on soma-memory
    │                 ▼
    │             soma-mcp        ──▷ Box<dyn KnowledgeBase>
    │
    └──▷ soma-python              ──▷ impl Filter, Step, Tool, PbtExecutor
             │                        « the only crate implementing four
             ▼                          foreign traits by calling into Python »
       python/soma/*.py

soma (the facade) sits outside this and re-exports ten of the thirteen.

D0 · The ownership spine

One screen. If you remember nothing else, remember this shape.

   User writes a Graph
        │
        ▼
   GraphSession                                soma-runtime/…/graph_session.rs:38
    ├──◆ Graph                                 « nodes + edges, no behaviour »
    │
    ├──◆ NodeCatalog                           « THE registry »
    │     ├──◆ HashMap<NodeId, NodeImpl>
    │     │      ├──◇ Arc<dyn Filter>          fit / forward
    │     │      └──◇ Arc<dyn Step>            poll -> Transition
    │     └──◇ Arc<dyn StateStore>             « shared across catalog clones »
    │
    ├──◇ Arc<dyn CacheStore>                   memory → local → action store
    │
    ├──◇ Arc<EventBus>
    │     ├──◆ broadcast::Sender<Event>        lossy: live subscribers
    │     └──◆ RwLock<Vec<Arc<dyn EventSink>>> lossless: JSONL to the run dir
    │
    ├──? Option<Arc<dyn DataStore>>            local / S3 / Zarr
    ├──? Option<Arc<dyn Transport>>            ┐ (!) two transport fields
    ├──◆ Vec<Arc<dyn Transport>>               ┘     D-04
    │
    └──? Option<EffectDriver>                  « present only if steps exist »
          ├──◆ Vec<Arc<dyn EffectHandler>>     llm · tools · sub-graph · sleep
          ├──◆ EffectJournal                   record once, replay forever
          │     ├──◇ Arc<dyn ActionCache>      kept forever
          │     └──◇ Arc<dyn BlobStore>        BLAKE3 CAS, evictable
          └──? Option<Arc<NodeCatalog>>        needed only for Transition::Spawn

   compile()  ──▷ ExecutionPlan  ──▷ LocalRunner ──▷ Context ──▷ run_node
                                                                    │
                                     output_key · compute_node · store_output

The ten types that carry the system

If you learn these, most of the rest follows.

#	Type	file:line	Why it matters
1	`Filter`	`soma-core/src/filter.rs:120`	`fit()` learns state, `forward()` transforms. Both independently cacheable. Everything pipeline-shaped is this
2	`Step`	`soma-core/src/step.rs:250`	`poll(ctx) -> Transition`. Everything agent-shaped is this. Holds no state between turns — history arrives through `StepCtx`
3	`NodeMeta`	`soma-core/src/node.rs:72`	The adapter that erases the Filter/Step distinction. `From<StepMeta>` sets `cacheable: false`, so “a step is not cacheable” is data, not a branch
4	`NodeCatalog`	`soma-runtime/src/node_catalog.rs:79`	One registry for both kinds, and the compiler’s `NodeRegistry`. Two registries joined by an adapter is what made `.compile()` skip step schemas
5	`Value`	`soma-core/src/value.rs:15`	Six variants, all `Arc`-backed, so `Clone` is a refcount bump
6	`CacheKey`	`soma-core/src/cache.rs:18`	`state = hash(config‖x‖y)`, `output = hash(config‖state‖input_hash)`. Downstream keys use input content, so an unchanged intermediate cuts off the rest of the graph
7	`ExecutionPlan`	`soma-compiler/src/plan.rs:19`	What the compiler produces and the executor walks. Recursive in four shapes; `children()` is the one traversal
8	`Context`	`soma-runtime/src/executor.rs:124`	The executor’s mutable state through the whole walk. `(!)` Also the biggest god object in the runtime
9	`Transition`	`soma-core/src/step.rs:43`	`Await` / `Spawn` / `Goto` / `Suspend` / `Done`. Deliberately not `#[non_exhaustive]` — every consumer must decide
10	`Effect` / `EffectJournal`	`soma-core/src/effect.rs:35`, `soma-runtime/src/effects/journal.rs:51`	An effect is data; the journal keys pure ones by content and impure ones by site. That is the whole durability story

The one distinction to internalize

A filter memoizes by content. A step journals by site.

A filter’s output is a function of its config, its state and its input, so an identical call anywhere can reuse the result. A step’s effects are not: asking a model the same question twice can give two answers, so an impure effect is keyed by where and when it happened — (run, node, turn, index) — recorded once and replayed on resume, never re-run.

Everything else about caching, resumption and reproducibility follows from that one sentence.

How a run actually happens

The narrative version of D2, for orientation. Every step links to the detail.

You build a Graph — nodes and edges, no behaviour. Nodes come in five structural kinds (Filter, SubGraph, Loop, Branch, Step); every behaviour is library code.
You register implementations in a NodeCatalog, which holds filters and steps side by side.
compile() (Execution) walks the graph, validates schemas between connected nodes, claims loop bodies and branch arms by dominance, wraps remote nodes, and returns an ExecutionPlan plus diagnostics.
LocalRunner::walk builds a Context from the plan and the topology — note topology, not plan order: input resolution follows predecessors, which is what makes a diamond work.
execute recurses over the plan. Each leaf reaches run_node, which is the one execution site for both kinds.
run_node resolves the input, fits state if needed, derives an output_key, checks the cache, and on a miss calls compute_node — the only place a filter’s forward and a step’s driver are told apart.
A step’s poll returns a Transition. If it is Await, the EffectDriver performs the effects on threads, consults the EffectJournal first, and calls poll again with the results. (D4)
Results are stored with provenance (Origin::Computed { node_id, run_id }), events are emitted to the EventBus, and a LocalTracker writes them to a run directory as JSONL.
Afterwards, RunReader and summarize turn that directory into a RunSummary, an ExperimentRecord lands in the pool, and .soma/HEAD advances — but only on success, and never inferred from a timestamp.

Remote execution replaces step 5 with a serialized plan over a WebSocket (D5); the executor itself does not change. Streaming replaces it with StreamRun, which composes the same three primitives per chunk — which is why a single-chunk stream and a plain forward produce identical cache keys.

Conventions this codebase keeps

Worth knowing before you write anything in it, because they are enforced by review rather than by the compiler.

#[non_exhaustive] is a decision, not a default. Data enums get it; control-flow enums every consumer must decide over (NodeOutcome, Transition, StreamMode) deliberately do not, so adding a variant breaks every match. The reason is in each doc comment. → Patterns
Unknown variants refuse, they do not guess. other => Err(…) naming the situation, at four sites.
Errors are typed at the edges, shared at the seams. Three error enums workspace-wide. → Decisions
Nothing is async. Zero async_trait. Concurrency is std::thread::scope.
Every crate opts into #![warn(missing_docs)].
Commits are Conventional Commits with a crate scope: feat(core): add Schema type.
cargo clippy --workspace -- -D warnings must pass. Ten #[allow] exist in ~70 000 lines, nine of them structural.

A caution about this section

These pages are hand-written and carry ~700 file:line anchors. Line numbers drift on the first edit above them.

The docs/scripts/check-anchors.mjs guard, wired into npm run check, verifies that every referenced file exists and that every named symbol still appears in it, and warns (without failing) when a line number has drifted more than 30 lines. That catches deletion and renaming — the failures that make a reference actively misleading — but it cannot tell you whether a description is still true.

When you find a claim here that is wrong, fix it. A reference nobody trusts is worse than no reference, which is exactly what happened to Architecture Review, now kept as a historical document.