02 — Caching and state: never compute the same thing twice

Generated from a notebook

This page is notebooks/02_caching_and_state.ipynb, executed, with its outputs. Run it yourself with jupyter lab notebooks/ after pip install 'somatize[viz]'. Interactive Plotly figures show here as the PNG the notebook also stores.

Every fit() and forward() in Soma is memoized in a persistent, content-addressable cache. The key ideas:

• State key = hash(filter identity + x + y) — same config trained on the same data returns the cached state, even in a different process, days later.
• Output key = hash(filter identity + state + input).
• By default every Graph() shares one on-disk cache at $SOMA_CACHE_DIR (or ~/.soma/cache). A crashed experiment re-run simply hits for everything that already completed — resume is a cache lookup, not a separate mechanism.

For this notebook we point SOMA_CACHE_DIR at a temp dir so we start cold.

import os, tempfile
os.environ["SOMA_CACHE_DIR"] = tempfile.mkdtemp(prefix="soma-nb02-")
print("cache root:", os.environ["SOMA_CACHE_DIR"])

cache root: /tmp/soma-nb02-36jdlsma

2.1 — fit() runs once per (config, data)

ExpensiveScaler counts how many times it actually fits. Watch the counter: two graphs, same config and data → one real fit.

from soma import Filter, Graph

FIT_CALLS = {"n": 0}

class ExpensiveScaler(Filter):
    _cache_version = "nb02-v1"   # explicit code-version pin (see 2.4)

    def __init__(self, factor=1.0, **kwargs):
        super().__init__(factor=factor, **kwargs)

    def fit(self, x, y=None):
        FIT_CALLS["n"] += 1        # pretend this takes hours
        return {"mean": sum(x) / len(x)}

    def forward(self, x, state):
        return [(v - state["mean"]) * self.factor for v in x]

data = [10.0, 20.0, 30.0]

for attempt in (1, 2):
    g = Graph()
    g.node("scaler", ExpensiveScaler(factor=2.0))
    g.fit(data)
    print(f"attempt {attempt}: total real fits = {FIT_CALLS['n']}")

attempt 1: total real fits = 1
attempt 2: total real fits = 1

The second graph found the state in the cache. This is exactly what happens when a crashed process is re-launched — completed work is served from disk, only the interrupted tail re-executes.

2.2 — Changing config (or data, or labels) invalidates

Any change to the filter’s parameters, the input data, or the labels produces a different key — no false hits, ever.

g = Graph()
g.node("scaler", ExpensiveScaler(factor=3.0))   # different factor
g.fit(data)
print("after factor=3.0:", FIT_CALLS["n"], "fits (new config → refit)")

g = Graph()
g.node("scaler", ExpensiveScaler(factor=2.0))
g.fit([1.0, 2.0, 3.0])                           # different data
print("after new data:  ", FIT_CALLS["n"], "fits (new data → refit)")

after factor=3.0: 2 fits (new config → refit)
after new data:   3 fits (new data → refit)

2.3 — Seeds are cache lines

Pass seed= to fit/forward and it is hashed into every key: each seed owns an independent, resumable cache line. A 5-seed experiment is 5 independent computations — losing seed 4 mid-run never costs you seeds 1–3.

before = FIT_CALLS["n"]
for seed in (1, 2, 1, 2):
    g = Graph()
    g.node("scaler", ExpensiveScaler(factor=2.0))
    g.fit([5.0, 6.0, 7.0], seed=seed)
print(f"4 seeded fits → {FIT_CALLS['n'] - before} real fits (one per distinct seed)")

4 seeded fits → 2 real fits (one per distinct seed)

2.4 — Code is part of the identity

A filter’s cache identity includes a code fingerprint, resolved through a ladder:

1. an explicit _cache_version = "..." class attribute — bump it to invalidate (survives refactors; recommended for filters whose helpers live in other modules),
2. otherwise a hash of the class source (editing the class invalidates),
3. last resort: a cloudpickle hash with a loud warning (not stable across Python versions — you’ll see it for classes defined in notebooks/REPLs, which is why ExpensiveScaler pins _cache_version).

Unhashable attributes raise a typed error instead of producing a broken key:

from soma import CacheConfigError

class BadFilter(Filter):
    _cache_version = "nb02-bad-v1"

    def __init__(self):
        super().__init__()
        self.handle = open(os.devnull)   # not JSON-serializable

    def forward(self, x, state):
        return x

try:
    Graph().node("bad", BadFilter())
except CacheConfigError as e:
    print("CacheConfigError:", str(e)[:120], "...")

CacheConfigError: attribute 'handle' of 'BadFilter' (type 'TextIOWrapper') cannot enter the cache key: TextIOWrapper. Prefix it with '_' t ...

2.5 — Managing the cache: soma cache

The store is two tables: tiny action records (what was computed, at what cost) and content-addressed blobs (the bytes, deduplicated). soma cache gc evicts blobs by value density (compute-time per byte) — and records are kept, so evicted entries are simply recomputed and re-fill the same address. Eviction can never lose correctness.

import subprocess, sys
print(subprocess.run([sys.executable, "-m", "soma._cache_cli", "cache", "stats"],
                     capture_output=True, text=True).stdout)

cache root      /tmp/soma-nb02-36jdlsma
action records  10
unique outputs  6 (6 on disk, 0 evicted)
CAS size        217 B
pinned          0
compute banked  0s

Other subcommands:

$ soma cache gc --max-size 20G     # evict low-value blobs down to 20 GiB
$ soma cache pin best-run <key>    # protect an action's outputs from GC
$ soma cache verify                # check blob integrity
$ soma cache purge-v1              # drop entries from pre-v2 layouts

2.6 — Opting out

Graph(cache="memory") restores a process-local, non-persistent cache (useful for tests). _cacheable = False on a filter excludes it from caching entirely; _deterministic = False marks a stochastic forward — it will never be served from cache unless the run pins a seed.