After CU22 — A block is a box, and a lane is inside the picture

Three things about the figure, found by looking at one: examples/07’s In[6], which is the cell that opens a composite.

An arrow that left the drawing

A routed edge takes a lane outside every box — a lane threaded between two of them is a lane that will cross a third the next time the layout moves — and the canvas was measured from the boxes. So the lane sat two pixels past the axis, and the arrows from the outer branches left the picture on one side and came back on the other.

The reason it survived a suite with a figure test in it is worth more than the fix: a routed edge is a path shape, and a path has no x0/y0. Every range check reads x0, so every range check skipped the only shapes that can leave the canvas. The test now parses the path.

A repeated block is a frame, not a word on every layer

Four encoder layers opened up were eight boxes each saying ×4: the count said eight times, and the block itself said none. Now a block of two or more layers is a frame around them with TransformerEncoderLayer ×4 on it, and a block that is a single layer keeps its ×N inline — a frame around one box says nothing a word could not.

An edge that comes down into a block ends on the block rather than on the layer inside it, because the frame’s header is where the count is written and an arrow through a label reads as neither. A skip comes in through the side, never touches the header, and goes on to the layer it really feeds: saying into the block there would lose the one thing a skip is about.

What runs several of itself at once

4 heads on a MultiheadAttention, drawn with plates behind the box. Read off num_heads and never inferred, and never drawn as separate boxes: torch packs the heads into one in_proj_weight and a reshape, so fx sees one operation and a hook sees one module. Four boxes wired together would be a graph nobody built — the same rule that makes a traced residual say how it was found.

The plates are capped at two however many lanes there are, because eight plates are a smudge and what says eight is the word. They go downwards, since the first layer of a block has its ×N immediately above it.

Opening a composite has to refine it, and once it deleted it

depth= opened a MultiheadAttention into out_proj, which is its only child and is never called: torch runs attention out of F.multi_head_attention_forward, handed the four tensors, and the module is only where they are kept. So no hook fired, the box disappeared, and depth=2 drew the encoder block’s norms and its feed-forward and no attention at all — which is a transformer figure with no transformer in it.

It was published as a :::caution saying the self-attention inside torch’s fused TransformerEncoderLayer is not surfaced at any depth. That diagnosis was wrong, and the way it was wrong is worth keeping: torch bails out of that fused path the moment any hook is attached anywhere under the layer, and architecture attaches one to everything it draws — so the fast path was never taken here and depth=1 had the attention all along. What the caution really described was depth=2, the setting the figure was generated at.

So depth gained a floor: MultiheadAttention is whole however deep it was asked for, because refining what has no parts that run has to answer the thing itself rather than nothing. Measured and not believed — the test hooks out_proj and watches it never fire, so whoever makes torch call it can delete the rule.

Why this was copied and not adopted

torchview does both of these already: expand_nested=True puts an expanded submodule in a dashed graphviz cluster, and roll=True collapses recursively used modules. Which is the confirmation that the shape is right, and the reason to take the idea rather than the tool.

What none of them has is the half this figure exists for. Netron, torchview and visualtorch draw a module tree; not one of them draws placement — no device, no host, no wave, no remote — and none has somewhere to put a health overlay. The original soma did not either, which is why CU19 had to invent it. Adopting graphviz would cost a system binary, lose plotly’s hover and the live view, and still leave the wave frames, the remote frames and the health channel to be written.

Questionnaire

Drawn (soma-python/tests/test_figure.py)

an edge routed around the drawing is still in the drawing, and the arrowheads too
a repeated block is a frame around its layers with the count on it, and not the count on each of them
and the frame holds every layer of the block and nothing else
a layer that runs identical lanes is drawn with them behind it
one lane is not several and draws nothing extra

What a node is made of (soma-python/tests/test_health.py)

a repeated block of several layers puts its count on the block
and a block that is one layer keeps its count inline
how many lanes a layer runs is read and never inferred
something that runs one lane says nothing about lanes
torch never calls the module inside an attention block, which is the measurement the floor rests on
and opening an encoder block shows its attention at every depth, with its lanes still on it