unconditional

unconditional

Encoder

Encoder(input_specification, pretrained=False, frozen=False, device='cuda' if is_available() else 'cpu', model_dtype=None)

Bases: EncodingMixin

Base class for all unconditional encoders.

Subclasses implement encode(). The base class forward() handles temporal flatten/unflatten — encode() receives tensors with the temporal axis flattened into the batch (images as (B*T, C, H, W), not (B, T, C, H, W)).

Initialize base encoder.

Parameters:

Name	Type	Description	Default
input_specification	EncoderInput	Structured input specification for this encoder	required
pretrained	bool	Whether to use pretrained weights	False
frozen	bool	Whether to freeze encoder weights	False
device	str | None	Device to place the encoder on	'cuda' if is_available() else 'cpu'
model_dtype	str | None	Precision string from experiment config (e.g. "bf16-mixed").	None

Source code in src/versatil/models/encoding/encoders/unconditional.py

def __init__(
    self,
    input_specification: EncoderInput,
    pretrained: bool = False,
    frozen: bool = False,
    device: str | None = "cuda" if torch.cuda.is_available() else "cpu",
    model_dtype: str | None = None,
):
    """Initialize base encoder.

    Args:
        input_specification: Structured input specification for this encoder
        pretrained: Whether to use pretrained weights
        frozen: Whether to freeze encoder weights
        device: Device to place the encoder on
        model_dtype: Precision string from experiment config (e.g. ``"bf16-mixed"``).
    """
    super().__init__(
        input_specification=input_specification,
        pretrained=pretrained,
        frozen=frozen,
        device=device,
        model_dtype=model_dtype,
    )

forward

forward(inputs)

Forward pass with temporal flatten/unflatten.

All input tensors must have a temporal dimension: images as (B, T, C, H, W), sequences as (B, T, S), vectors as (B, T, D). The temporal dimension is flattened into batch before calling encode(), then restored after.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Tensor]	Dict mapping input_keys to tensors with temporal dimension.	required

Returns:

Type	Description
dict[str, Tensor]	Dict mapping feature names to feature tensors with temporal dimension.

Source code in src/versatil/models/encoding/encoders/unconditional.py

def forward(self, inputs: dict[str, torch.Tensor]) -> dict[str, torch.Tensor]:
    """Forward pass with temporal flatten/unflatten.

    All input tensors must have a temporal dimension: images as (B, T, C, H, W),
    sequences as (B, T, S), vectors as (B, T, D). The temporal dimension is
    flattened into batch before calling encode(), then restored after.

    Args:
        inputs: Dict mapping input_keys to tensors with temporal dimension.

    Returns:
        Dict mapping feature names to feature tensors with temporal dimension.
    """
    self._validate_inputs(inputs)
    inputs, batch_size, temporal_length = self._flatten_temporal(inputs)
    outputs = self.encode(inputs)
    return self._unflatten_temporal(outputs, batch_size, temporal_length)

encode abstractmethod

encode(inputs)

Encode temporally flattened inputs.

Subclasses implement this. Images are (BT, C, H, W), tokenized text is (BT, S), proprioceptive state is (B*T, D); forward() flattens the temporal axis into the batch before dispatching here.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Tensor]	Dict mapping input_keys to temporally flattened tensors.	required

Returns:

Type	Description
dict[str, Tensor]	Dict mapping feature names to feature tensors.

Source code in src/versatil/models/encoding/encoders/unconditional.py

@abstractmethod
def encode(self, inputs: dict[str, torch.Tensor]) -> dict[str, torch.Tensor]:
    """Encode temporally flattened inputs.

    Subclasses implement this. Images are (B*T, C, H, W),
    tokenized text is (B*T, S), proprioceptive state is (B*T, D);
    ``forward()`` flattens the temporal axis into the batch before
    dispatching here.

    Args:
        inputs: Dict mapping input_keys to temporally flattened tensors.

    Returns:
        Dict mapping feature names to feature tensors.
    """
    raise NotImplementedError