hdf5

hdf5

Abstract dataset schema for HDF5-based datasets.

Hdf5DatasetSchema

Hdf5DatasetSchema(hdf5_paths, zarr_path, metadata, dataset_type)

Bases: DatasetSchema

Abstract schema for HDF5-based datasets.

HDF5 datasets store all data (observations, images, actions) in HDF5 files. Subclasses define the HDF5 structure and extraction logic via extract_episode.

Initialize the HDF5 dataset schema.

Parameters:

Name	Type	Description	Default
hdf5_paths	list[str]	List of paths to HDF5 files	required
zarr_path	str	Path to save/load the zarr file	required
metadata	DatasetMetadata	Metadata to use for creating the zarr store from the raw data.	required
dataset_type	str	Type of dataset used by the schema (e.g., 'libero', 'tso', 'metaworld')	required

Source code in src/versatil/data/raw/schemas/hdf5.py

def __init__(
    self,
    hdf5_paths: list[str],
    zarr_path: str,
    metadata: DatasetMetadata,
    dataset_type: str,
):
    """Initialize the HDF5 dataset schema.

    Args:
        hdf5_paths: List of paths to HDF5 files
        zarr_path: Path to save/load the zarr file
        metadata: Metadata to use for creating the zarr store from the raw data.
        dataset_type: Type of dataset used by the schema (e.g., 'libero', 'tso', 'metaworld')
    """
    super().__init__(
        zarr_path=zarr_path,
        metadata=metadata,
        dataset_type=dataset_type,
    )
    self.hdf5_paths = hdf5_paths

get_demo_names abstractmethod

get_demo_names(hdf5_path)

Get list of demonstration/episode names in the specified HDF5 file.

Parameters:

Name	Type	Description	Default
hdf5_path	str	Path to the HDF5 file	required

Returns:

Type	Description
list[str]	List of demo identifiers as strings.

Source code in src/versatil/data/raw/schemas/hdf5.py

@abc.abstractmethod
def get_demo_names(self, hdf5_path: str) -> list[str]:
    """Get list of demonstration/episode names in the specified HDF5 file.

    Args:
        hdf5_path: Path to the HDF5 file

    Returns:
        List of demo identifiers as strings.
    """
    raise NotImplementedError("Subclasses must implement get_demo_names")

extract_episode abstractmethod

extract_episode(demo_group)

Extract episodic data from a single HDF5 group.

Source code in src/versatil/data/raw/schemas/hdf5.py

@abc.abstractmethod
def extract_episode(
    self,
    demo_group: h5py.Group,
) -> dict[str, np.ndarray]:
    """Extract episodic data from a single HDF5 group."""
    raise NotImplementedError("Subclasses must implement extract_episode")