csv

csv

Abstract dataset schema for CSV-based datasets.

CsvDatasetSchema

CsvDatasetSchema(dataset_folders, zarr_path, episode_filename, metadata, dataset_type)

Bases: DatasetSchema

Abstract schema for CSV-based datasets.

CSV datasets store observations in tabular format. Images are stored as image path values.

Initialize the CSV dataset schema.

Parameters:

Name	Type	Description	Default
dataset_folders	list[str]	List of dataset folder paths	required
zarr_path	str	Path to save/load the zarr file	required
episode_filename	str	Name and format of the episode CSV data file in each folder	required
metadata	DatasetMetadata	Metadata to use for creating the zarr store from the raw data.	required
dataset_type	str	Type of dataset (e.g., 'libero', 'tso', 'metaworld')	required

Source code in src/versatil/data/raw/schemas/csv.py

def __init__(
    self,
    dataset_folders: list[str],
    zarr_path: str,
    episode_filename: str,
    metadata: DatasetMetadata,
    dataset_type: str,
):
    """Initialize the CSV dataset schema.

    Args:
        dataset_folders: List of dataset folder paths
        zarr_path: Path to save/load the zarr file
        episode_filename: Name and format of the episode CSV data file in each folder
        metadata: Metadata to use for creating the zarr store from the raw data.
        dataset_type: Type of dataset (e.g., 'libero', 'tso', 'metaworld')
    """
    super().__init__(
        zarr_path=zarr_path,
        metadata=metadata,
        dataset_type=dataset_type,
    )
    self.dataset_folders = dataset_folders
    self.dataset_filename = episode_filename

extract_episode abstractmethod

extract_episode(episode)

Extract all data from an episode DataFrame.

Source code in src/versatil/data/raw/schemas/csv.py

@abc.abstractmethod
def extract_episode(
    self,
    episode: pd.DataFrame,
) -> dict[str, np.ndarray]:
    """Extract all data from an episode DataFrame."""
    ...