zarr_meta

zarr_meta

Raw dataset metadata for creating the dataset zarr store for fast parallel access at training time.

dtype across all classes uses the zarr v3 type convention. zarr v3 allowed dtypes are defined here https://zarr-specs.readthedocs.io/en/latest/v3/data-types/index.html

DatasetMetadata dataclass

DatasetMetadata(observations=dict(), precomputed_actions=dict())

Raw dataset metadata needed for creating the dataset zarr store.

This class aggregates all observation and action metadata from the raw dataset. Generic validation happens at instantiation via post_init. Dataset-specific validation should be performed by the DatasetSchema.

Attributes:

Name	Type	Description
observations	dict[str, ObservationMetadata | CameraMetadata]	Dict of all observation metadata, indexed by zarr store key. Values are CameraMetadata, or ObservationMetadata.
precomputed_actions	dict[str, PrecomputedActionMetadata]	Dict of precomputed action metadata, indexed by zarr store key. Values are PrecomputedActionMetadata.

cameras property

cameras

Get all camera observations.

depth_cameras property

depth_cameras

Get all depth camera observations.

rgb_cameras property

rgb_cameras

Get all RGB camera observations.

position_observations property

position_observations

Get all position observations.

orientation_observations property

orientation_observations

Get all orientation observations.

gripper_observations property

gripper_observations

Get all gripper state observations.

proprioceptive_observations property

proprioceptive_observations

Get all proprioceptive observations (position, orientation, gripper).

custom_observations property

custom_observations

Get custom observations (not position, orientation, gripper, or camera).

position_actions property

position_actions

Get all precomputed position actions.

orientation_actions property

orientation_actions

Get all precomputed orientation actions.

gripper_actions property

gripper_actions

Get all precomputed gripper actions.

custom_actions property

custom_actions

Get custom precomputed actions (not position, orientation, or gripper).

__post_init__

__post_init__()

Validate metadata consistency and resolve OmegaConf interpolation keys.

Source code in src/versatil/data/raw/zarr_meta.py

def __post_init__(self):
    """Validate metadata consistency and resolve OmegaConf interpolation keys."""
    self.observations = resolve_dict_keys(self.observations)
    self.precomputed_actions = resolve_dict_keys(self.precomputed_actions)
    obs_keys = set(self.observations.keys())
    action_keys = set(self.precomputed_actions.keys())
    overlap = obs_keys & action_keys
    if overlap:
        raise ValueError(f"Keys cannot be both observations and actions: {overlap}")

    camera_keys = [
        k for k, v in self.observations.items() if isinstance(v, CameraMetadata)
    ]
    if len(camera_keys) != len(set(camera_keys)):
        raise ValueError(f"Duplicate camera keys found: {camera_keys}")

    validate_camera_metadata_keys(self.cameras)

get_all_keys

get_all_keys()

Get all zarr keys (observations + actions).

Source code in src/versatil/data/raw/zarr_meta.py

def get_all_keys(self) -> list[str]:
    """Get all zarr keys (observations + actions)."""
    return list(self.observations.keys()) + list(self.precomputed_actions.keys())

get_camera_keys

get_camera_keys()

Get list of all camera keys.

Source code in src/versatil/data/raw/zarr_meta.py

def get_camera_keys(self) -> list[str]:
    """Get list of all camera keys."""
    return list(self.cameras.keys())

get_proprio_dimension

get_proprio_dimension()

Get total proprioceptive observation dimension.

Source code in src/versatil/data/raw/zarr_meta.py

def get_proprio_dimension(self) -> int:
    """Get total proprioceptive observation dimension."""
    return sum(obs.dimension for obs in self.proprioceptive_observations.values())

get_gripper_dimension

get_gripper_dimension()

Get gripper state dimension.

Source code in src/versatil/data/raw/zarr_meta.py

def get_gripper_dimension(self) -> int:
    """Get gripper state dimension."""
    return sum(obs.dimension for obs in self.gripper_observations.values())

has_precomputed_actions

has_precomputed_actions()

Check if dataset has any precomputed actions.

Source code in src/versatil/data/raw/zarr_meta.py

def has_precomputed_actions(self) -> bool:
    """Check if dataset has any precomputed actions."""
    return len(self.precomputed_actions) > 0

get_precomputed_action

get_precomputed_action(key)

Get a specific precomputed action by key.

Source code in src/versatil/data/raw/zarr_meta.py

def get_precomputed_action(self, key: str) -> PrecomputedActionMetadata | None:
    """Get a specific precomputed action by key."""
    return self.precomputed_actions.get(key)

get_observation

get_observation(key)

Get a specific observation by key.

Source code in src/versatil/data/raw/zarr_meta.py

def get_observation(self, key: str) -> ObservationMetadata | CameraMetadata | None:
    """Get a specific observation by key."""
    return self.observations.get(key)

has_observation

has_observation(key)

Check if an observation exists.

Source code in src/versatil/data/raw/zarr_meta.py

def has_observation(self, key: str) -> bool:
    """Check if an observation exists."""
    return key in self.observations