Skip to content

conditioning

conditioning

Conditioning cache for static context (observations, encoder features).

Conditioning information does not change between forward calls — whether consumed via cross-attention, joint attention, or prefix concatenation. The cache is computed once before generation starts and reused unchanged across all forward calls.

ConditioningLayerCache dataclass 

ConditioningLayerCache(keys, values, queries=None)

Per-layer K/V cache for static conditioning (computed once, reused).

Stores precomputed projections for conditioning information that does not change between forward calls. Populated once before generation starts.

Attributes:

Name Type Description
keys Tensor

Precomputed keys (B, kv_heads, S, head_dim).
values Tensor

Precomputed values (B, kv_heads, S, head_dim).
queries Tensor | None

Precomputed queries (B, heads, S, head_dim). Populated when conditioning participates bidirectionally (both streams attend to each other). None when conditioning is only attended to.

ConditioningCache dataclass 

ConditioningCache(layers)

Multi-layer conditioning cache for static context.

Attributes:

Name Type Description
layers list[ConditioningLayerCache | None]

One ConditioningLayerCache (or None) per decoder layer.

__getitem__ 

__getitem__(index)

Get cache layer by slicing.

Source code in src/versatil/models/layers/transformer/cache/conditioning.py 
46
47
48
def __getitem__(self, index: int) -> ConditioningLayerCache | None:
    """Get cache layer by slicing."""
    return self.layers[index]

CacheableLayer

Bases: Protocol

Layer that can precompute conditioning K/V projections.

precompute_conditioning_kv 

precompute_conditioning_kv(encoded_features)

Project encoded features to cacheable conditioning keys and values.

Source code in src/versatil/models/layers/transformer/cache/conditioning.py 
55
56
57
58
59
def precompute_conditioning_kv(
    self, encoded_features: torch.Tensor
) -> ConditioningLayerCache | None:
    """Project encoded features to cacheable conditioning keys and values."""
    ...

precompute_conditioning 

precompute_conditioning(layers, encoded_features)

Precompute conditioning K/V for all layers.

Calls each layer's precompute_conditioning_kv to project encoder features through cross-attention K/V weights once, for reuse across forward calls.

Parameters:

Name Type Description Default
layers Iterable[CacheableLayer]

Decoder layers satisfying CacheableLayer protocol.

 required
encoded_features Tensor

Encoder features (B, memory_length, D).

 required

Returns:

Type Description
ConditioningCache

ConditioningCache with one ConditioningLayerCache per layer.
Source code in src/versatil/models/layers/transformer/cache/conditioning.py 
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def precompute_conditioning(
    layers: Iterable[CacheableLayer],
    encoded_features: torch.Tensor,
) -> ConditioningCache:
    """Precompute conditioning K/V for all layers.

    Calls each layer's precompute_conditioning_kv to project encoder features
    through cross-attention K/V weights once, for reuse across forward calls.

    Args:
        layers: Decoder layers satisfying CacheableLayer protocol.
        encoded_features: Encoder features (B, memory_length, D).

    Returns:
        ConditioningCache with one ConditioningLayerCache per layer.
    """
    return ConditioningCache(
        layers=[layer.precompute_conditioning_kv(encoded_features) for layer in layers]
    )