Module delta

Delta encoding and decoding for vector compression.

Instead of storing every embedding in full, CVX stores keyframes (full vectors) at regular intervals and deltas (sparse changes) between them. This is analogous to video compression (I-frames + P-frames).

A delta is sparse: only dimensions that changed by more than ε are stored. For slowly-drifting embeddings, most dimensions don’t change between updates, yielding 3-10x compression.

Example

use cvx_ingest::delta::{DeltaEncoder, DeltaDecoder};

let encoder = DeltaEncoder::new(10, 0.001); // keyframe every 10, threshold 0.001

let v1 = vec![1.0, 2.0, 3.0];
let v2 = vec![1.0, 2.1, 3.0]; // only dim 1 changed above ε

let entry1 = encoder.encode(0, 1000, &v1, None);
assert!(entry1.is_keyframe()); // first vector is always a keyframe

let entry2 = encoder.encode(1, 2000, &v2, Some(&v1));
assert!(!entry2.is_keyframe());
assert_eq!(entry2.nnz(), 1); // only 1 dimension stored

// Reconstruct v2 from keyframe + delta
let reconstructed = DeltaDecoder::apply(&v1, &entry2);
assert!((reconstructed[1] - 2.1).abs() < 1e-7);

Structs

DeltaDecoder

Reconstructs full vectors from keyframes and delta chains.

DeltaEncoder

Encodes vectors into keyframe + delta sequences.

Functions

compression_ratio

Compute the compression ratio: full_size / delta_size.