somatize_store/lib.rs
1//! Where what is worth keeping is kept: bytes by content, and names that point
2//! at them. The hole the core left, plugged with directories, hashes and atomic
3//! renames.
4//!
5//! | map | key | what it is for |
6//! |---|---|---|
7//! | **blobs** | the digest of the content | weights, artifacts, intermediate data |
8//! | **names** | whatever you call it | a cache key, an artifact's id |
9//!
10//! Two and not one because **a cache key is not a content hash**: it is the hash
11//! of the recipe, known *before* the value it names exists. Content addressing
12//! alone could not answer *do I already have what this recipe produces?*
13//!
14//! A blob is written once and never changes; a binding is a small record written
15//! by atomic rename. That is what lets a network folder be shared by everyone at
16//! once with no lock server, and it turned out to work unchanged against a
17//! bucket.
18//!
19//! | who | what it keeps | under what name |
20//! |---|---|---|
21//! | a worker's `Serving::store` | artifacts, so a catalog is not sent twice | `artifact:<kind>:<id>` |
22//! | [`Cache`] | what a node produced | `value:<key>` |
23//! | [`Recorder`] | what happened, one record per `forward` | `run/<id>/<n>` |
24//!
25//! [`Cache`] is the [`Keeper`](somatize_core::Keeper) and [`Recorder`] the
26//! [`Watcher`](somatize_core::Watcher) — the same division from both ends. A
27//! queryable index is **derived** from these records and can be thrown away;
28//! making it the truth would mean a single writer, which is where NFS breaks.
29
30#![forbid(unsafe_code)]
31#![warn(missing_docs)]
32
33mod cache;
34mod digest;
35mod local;
36mod recorder;
37#[cfg(feature = "s3")]
38mod s3;
39mod store;
40
41pub use cache::{Cache, bytes_of, name_of, value_of};
42pub use digest::Digest;
43pub use local::Local;
44pub use recorder::Recorder;
45#[cfg(feature = "s3")]
46pub use s3::{Bucket, Credentials, UrlStyle};
47pub use store::{Bound, Meta, Store, StoreError};