Skip to content

schema_paths

schema_paths

Dataset schema path overrides for offline explainability runs.

resolve_dataset_schema_for_explanation 

resolve_dataset_schema_for_explanation(schema, data_path_override, zarr_cache_directory)

Return the dataset schema used by an offline explanation run.

Parameters:

Name Type Description Default
schema DatasetSchema

Checkpoint task schema. Its metadata remains the structure contract for offline explanation data.

 required
data_path_override str | list[str] | None

Optional offline input location to explain instead of the data path stored in the checkpoint task config. None returns schema unchanged. A single path ending in .zarr is an existing replay buffer and is sampled directly. A non-zarr path is raw data in the same dataset schema as the checkpoint. A list is only for raw schemas that already support multiple inputs, such as CSV dataset_folders or HDF5 hdf5_paths; multiple zarr paths are rejected.

 required
zarr_cache_directory Path

Directory where raw override data is converted to a zarr replay buffer before sampling episodic windows.

 required

Returns:

Type Description
DatasetSchema

Schema to pass to the existing zarr creation and episodic dataset
DatasetSchema

pipeline.

Raises:

Type Description
ValueError

If the override is empty, points to missing input data, uses a zarr path that does not exist, or does not match the checkpoint schema storage type.
Source code in src/versatil/explainability/sources/schema_paths.py 
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def resolve_dataset_schema_for_explanation(
    schema: DatasetSchema,
    data_path_override: str | list[str] | None,
    zarr_cache_directory: Path,
) -> DatasetSchema:
    """Return the dataset schema used by an offline explanation run.

    Args:
        schema: Checkpoint task schema. Its metadata remains the structure
            contract for offline explanation data.
        data_path_override: Optional offline input location to explain instead
            of the data path stored in the checkpoint task config. ``None``
            returns ``schema`` unchanged. A single path ending in ``.zarr`` is
            an existing replay buffer and is sampled directly. A non-zarr path
            is raw data in the same dataset schema as the checkpoint. A list is
            only for raw schemas that already support multiple inputs, such as
            CSV ``dataset_folders`` or HDF5 ``hdf5_paths``; multiple zarr paths
            are rejected.
        zarr_cache_directory: Directory where raw override data is converted to
            a zarr replay buffer before sampling episodic windows.

    Returns:
        Schema to pass to the existing zarr creation and episodic dataset
        pipeline.

    Raises:
        ValueError: If the override is empty, points to missing input data, uses
            a zarr path that does not exist, or does not match the checkpoint
            schema storage type.
    """
    if data_path_override is None:
        return schema

    data_paths = _normalize_data_paths(data_path_override=data_path_override)
    if _is_zarr_override(data_paths=data_paths):
        return _resolve_zarr_override(schema=schema, zarr_path=data_paths[0])

    for data_path in data_paths:
        if not data_path.exists():
            raise ValueError(f"data_path_override path does not exist: {data_path}")

    override_schema = deepcopy(schema)
    override_schema.zarr_path = str(zarr_cache_directory / OFFLINE_DATASET_ZARR_NAME)

    if isinstance(override_schema, CsvDatasetSchema):
        override_schema.dataset_folders = [str(data_path) for data_path in data_paths]
    elif isinstance(override_schema, Hdf5DatasetSchema):
        override_schema.hdf5_paths = [str(data_path) for data_path in data_paths]
    elif isinstance(override_schema, LeRobotDatasetSchemaV30):
        _replace_lerobot_dataset_path(
            schema=override_schema,
            data_paths=data_paths,
        )
    elif isinstance(override_schema, SyntheticSchema):
        raise ValueError(
            "data_path_override cannot point to raw files for SyntheticSchema. "
            "Pass an existing .zarr path instead."
        )
    else:
        raise ValueError(
            "data_path_override is unsupported for schema type "
            f"{type(schema).__name__}."
        )

    return override_schema