12 — The campaign: using what you already recorded

Notebook 11 fixed four things at once and the model got better. That is the least informative possible result: it does not say which fix mattered, and the honest answer turns out to be one you would not guess.

Every run in notebooks 10 and 11 recorded itself — its architecture, its conclusion, and the change from whatever it descended from. This notebook uses that: four variants, one lineage, and a question answered from the record rather than from memory.

import warnings

import numpy as np
import torch
import torch.nn as nn
import plotly.io as pio

import soma
from soma import Graph

pio.renderers.default = "plotly_mimetype+png"
pio.renderers["png"].scale = 2
pio.renderers["png"].width = 950
warnings.filterwarnings("ignore", message="Full backward hook is firing")

# Imported from a module, not defined in a cell — and that is load
# bearing. Graph.load() resolves every node through importlib, so a
# class that exists only in a kernel has no import path and a saved
# checkpoint cannot be reopened. See the last section.
from campaign import DualViewEncoder, Head
import campaign

X, y = campaign.make_windows(512, seed=0)
F = campaign.features(X)
F = campaign.standardize(F[:384], F)
xtr = torch.tensor(F[:384], dtype=torch.float32)
ytr = torch.tensor(y[:384])
xva = torch.tensor(F[384:], dtype=torch.float32)
yva = y[384:]

12.1 — The variant runner

Twenty lines, no magic. Each variant is a graph, a short training loop, and a tracked run whose params record the switches — which is what makes the difference between two runs machine-readable later.

def train_variant(name, *, hypothesis=None, **cfg):
    torch.manual_seed(0)
    g = Graph()
    g.node("encoder", DualViewEncoder(**cfg))
    g.node("head", Head())
    g.connect("encoder", "head")
    g.materialize(xtr)
    g.train()
    g.make_optimizer(torch.optim.Adam, lr=1e-2)

    with g.track_run(name, tags=["campaign"], params=cfg,
                     hypothesis=hypothesis) as run:
        for epoch in range(12):
            order = torch.randperm(384)
            for i in range(0, 384, 64):
                idx = order[i:i + 64]
                with g.context() as ctx:
                    g.zero_grad()
                    out, _ = g.forward(xtr[idx])
                    g.backward(ctx, nn.functional.cross_entropy(out, ytr[idx]))
                g.step(ctx)
            g.eval()
            with torch.no_grad():
                acc = campaign.accuracy(np.asarray(g.forward(xva)), yva)
            g.train()
            run.log("val_acc", acc, step=epoch)
    print(f"{name:16s} val_acc {acc:.3f}")
    return run.id, acc, g

AS_FOUND = dict(leak_wiring=True, dead_bias=True, starve_context=True,
                trunk_gain=0.30, depth=5)

12.2 — The baseline

.soma/HEAD points at whichever run finished last, and the next run descends from it automatically. Nothing to wire up.

soma.detach()   # this notebook starts its own research line

base, base_acc, _ = train_variant(
    "as-found",
    hypothesis="the dual-view encoder earns its second branch",
    **AS_FOUND,
)
print("HEAD ->", soma.head())

as-found         val_acc 0.664
HEAD -> run_20260804T003534_cb81

12.3 — Fix the wiring

HEAD is already on the baseline, so this run records itself as its child, with the parameter change as the edge between them.

wiring, wiring_acc, _ = train_variant(
    "fix-wiring", **{**AS_FOUND, "leak_wiring": False})

fix-wiring       val_acc 0.672

Nothing. The leakage was real — notebook 11 measured a CKA of 0.90 between two branches that should share nothing — and fixing it bought almost exactly zero.

12.4 — Fix the trunk instead

This one has to be a sibling of the baseline, not a child of fix-wiring. Rewind HEAD first.

Soma will not guess this for you. It never infers a parent from timestamps: “the run before this one” is a different claim from “the run this one came from”, and one false edge would poison every delta computed downstream of it.

soma.checkout(base)
trunk, trunk_acc, _ = train_variant(
    "fix-trunk", **{**AS_FOUND, "trunk_gain": 1.0})

fix-trunk        val_acc 0.648

Also nothing. Two plausible fixes, two null results.

12.5 — Both at once

soma.checkout(wiring)
both, both_acc, g_best = train_variant(
    "wiring+trunk", **{**AS_FOUND, "leak_wiring": False, "trunk_gain": 1.0})

print(f"\nwiring alone  {wiring_acc - base_acc:+.3f}")
print(f"trunk alone   {trunk_acc - base_acc:+.3f}")
print(f"both together {both_acc - base_acc:+.3f}")

wiring+trunk     val_acc 0.969

wiring alone  +0.008
trunk alone   -0.016
both together +0.305

There it is. Neither change does anything alone; together they are worth thirty points.

Which makes sense once you see it: the second view carried information the first did not (notebook 10), and the contracting trunk meant no gradient reached the branches to make use of it. Fixing either one leaves the other as the binding constraint. This is exactly the result that a one-variable-at-a-time sweep reports as “neither helps”.

12.6 — What the pool recorded

Nothing below was written by hand. Every row is a by-product of having run the thing.

df = soma.experiments_dataframe()
df[["name", "parent", "metric_val_acc"]]

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

	name	parent	metric_val_acc
0	as-found	NaN	0.664062
1	fix-wiring	run_20260804T003534_cb81	0.671875
2	fix-trunk	run_20260804T003534_cb81	0.648438
3	wiring+trunk	run_20260804T003534_46f2	0.968750

for rec in soma.experiments():
    move = (rec.get("derivation") or {}).get("summary")
    print(f"{rec['name']:16s} {move or '(root of this line)'}")

as-found         (root of this line)
fix-wiring       encoder reconfigured; leak_wiring: true → false ⇒ val_acc +0.0078
fix-trunk        encoder reconfigured; trunk_gain: 0.3 → 1.0 ⇒ val_acc −0.0156
wiring+trunk     encoder reconfigured; trunk_gain: 0.3 → 1.0 ⇒ val_acc +0.2969

Those summaries are the edges. A tree of runs tells you what you ran; a tree of runs plus the change on every edge tells you what you tried.

tree = soma.lineage(both)
print(f"{'root':>10s}  {tree['ancestors'][0]['name'] if tree['ancestors'] else tree['focus']['name']}")
for a in tree["ancestors"]:
    print(f"{'':>10s}  {a['id']}  {a['name']}")
print(f"{'focus':>10s}  {tree['focus']['id']}  {tree['focus']['name']}")

# The whole tree, from the root, indented by depth.
root_id = tree["ancestors"][0]["id"] if tree["ancestors"] else both
whole = soma.lineage(root_id)
print(f"\n{whole['focus']['name']}")
for node in whole["descendants"]:
    move = (node["record"].get("derivation") or {}).get("summary", "")
    print(f"{'  ' * node['depth']}└─ {node['record']['name']:16s} {move}")

      root  as-found
            run_20260804T003534_cb81  as-found
            run_20260804T003534_46f2  fix-wiring
     focus  run_20260804T003535_1f93  wiring+trunk

as-found
  └─ fix-wiring       encoder reconfigured; leak_wiring: true → false ⇒ val_acc +0.0078
    └─ wiring+trunk     encoder reconfigured; trunk_gain: 0.3 → 1.0 ⇒ val_acc +0.2969
  └─ fix-trunk        encoder reconfigured; trunk_gain: 0.3 → 1.0 ⇒ val_acc −0.0156

fix-trunk hangs off the baseline as a sibling, exactly as intended — the checkout in 12.4 is what put it there.

12.7 — Comparing two branches that never met

A recorded derivation only exists on a parent→child edge. fix-wiring and fix-trunk are siblings, so there is no edge between them — but they are the two runs you most want to compare.

move = soma.diff(wiring, trunk)
for change in move["changes"]:
    print(change)
print()
for metric, delta in move["metric_delta"].items():
    print(f"{metric}: {delta['before']:.3f} -> {delta['after']:.3f} ({delta['delta']:+.3f})")

{'change': 'NodeReconfigured', 'node': 'encoder', 'from_hash': '14cd9c99b3b2567d8a3d5edbf7a900bfad3346f42853e0d0405319943a1be040', 'to_hash': '81fd881ac22187d94c5581c4c68eaf71d179b693895cc6c69067103bd6c9e930'}
{'change': 'ParamChanged', 'key': 'leak_wiring', 'from': False, 'to': True}
{'change': 'ParamChanged', 'key': 'trunk_gain', 'from': 0.3, 'to': 1.0}

val_acc: 0.672 -> 0.648 (-0.023)

12.8 — Retain the finding

The whole point of noticing an interaction is that nobody has to notice it twice. A conclusion is appended as its own journal line — the original record is never rewritten — and it is indexed for retrieval like any other text.

amendment = soma.record_conclusion(
    trunk,
    "Fixing the trunk alone buys nothing, and so does fixing the wiring alone. "
    "Together they are worth +0.30: the second view carries information the "
    "first does not, and the contractive trunk stopped any gradient reaching "
    "the branches to exploit it. Do not evaluate these two changes separately.",
    tags=["interaction", "dead-end-alone"],
)
print("amendment:", amendment)

amendment: amend_18c8730931f2ca1f

12.9 — Getting it back

This is the query the pool exists for. Note that the dead ends rank: importance is floored for any run that failed, crashed or regressed and carries a conclusion, because not repeating a dead end saves as much time as repeating a success.

for hit in soma.find_similar("trunk gradient does not reach the branches", limit=3):
    rec = hit["record"]
    print(f"{hit['score']:.2f}  {rec['name']}")
    print(f"      {hit['why']}")
    if rec.get("notes"):
        print(f"      note: {rec['notes'][:96]}...")
    print()

0.87  amendment to run_20260804T003534_7cbe
      score 0.87 (text 1.00, structure 0.00, recency 1.00, importance 0.50)
      note: Fixing the trunk alone buys nothing, and so does fixing the wiring alone. Together they are wort...

0.48  as-found
      score 0.48 (text 0.17, structure 0.00, recency 1.00, importance 0.70)

0.42  wiring+trunk
      score 0.42 (text 0.02, structure 0.00, recency 1.00, importance 0.80)

# Or search by architecture rather than by words: "what else looks
# structurally like this run?"
for hit in soma.find_similar(like_run=both, limit=3):
    print(f"{hit['score']:.2f}  {hit['record']['name']:16s} "
          f"structure {hit['components']['structural']:.2f}")

0.93  fix-wiring       structure 1.00
0.93  wiring+trunk     structure 1.00
0.90  as-found         structure 1.00

12.10 — What the cache paid for, and reopening the winner

Graph.load resolves each node’s class_path through importlib. campaign.DualViewEncoder resolves; a class defined in a cell above would be __main__.DualViewEncoder, which does not — which is why this campaign’s filters live in a module.

g_best.freeze()
g_best.save("pulse_best.somack")

restored = Graph.load("pulse_best.somack")
restored.eval()
with torch.no_grad():
    restored_acc = campaign.accuracy(np.asarray(restored.forward(xva)), yva)
print(f"original {both_acc:.3f}   restored {restored_acc:.3f}")

original 0.969   restored 0.969

print(soma.reindex(), "records rebuilt from the run directories")

5 records rebuilt from the run directories

The run directories are the source of truth; the journal is an index over them. Delete experiments.jsonl and soma kb reindex puts it back, lineage and all.

Where this goes

Everything in this notebook is also an MCP tool — kb_find_similar, kb_lineage, kb_diff, kb_record_conclusion, kb_branch_from — so an agent runs the same loop against the same pool, and reads the same conclusions you just wrote.

Experiment Pool — the design, the scoring formula, and what is deliberately not built yet
Knowledge Base — the API
soma report <run_id> packages any of these runs as one shareable HTML file