dataloader

dataloader

DataLoaderConfig dataclass

DataLoaderConfig(preload_data_in_memory=False, recreate_zarr_on_missing_keys=False, batch_size=64, num_workers=16, shuffle=True, image_norm_type=value, depth_norm_type=value, kinematics_norm_type=value, winsorize_depth=True, depth_winsorize_quantiles=(0.01, 0.99), winsorize_kinematics=True, kinematics_winsorize_quantiles=(0.01, 0.99), clamp_kinematics_range=True, min_kinematics_std=0.01, min_kinematics_range=0.01, tokenization=TokenizationConfig(), color_augmentation=AugmentationPipelineConfig(), spatial_augmentation=AugmentationPipelineConfig(), skip_initial_episode_steps=0, downsample_factor=1, action_backward_shift=0, trailing_padded_actions=None, val_ratio=0.1, total_ratio=1.0, action_sample_size=2048)

Hydra config for dataset loading, normalization, augmentation, and sampling behavior.

Attributes:

Name	Type	Description
preload_data_in_memory	bool	Whether to preload the entire zarr into RAM, speeds up training considerably but works only for small datasets.
recreate_zarr_on_missing_keys	bool	Whether a zarr store lacking keys required by this task may be deleted and rebuilt from the raw sources. Off by default so a wrong task configuration cannot destroy an expensive store.
batch_size	int	Batching.
num_workers	int	Dataloader worker process count.
shuffle	bool	Whether training samples are shuffled each epoch.
image_norm_type	str	Data processing.
depth_norm_type	str	Normalization mode for depth images.
kinematics_norm_type	str	Normalization mode for kinematic signals.
winsorize_depth	bool	Whether depth values are clipped to quantile bounds before fitting.
depth_winsorize_quantiles	tuple[float, float]	Lower and upper quantiles clipping depth values.
winsorize_kinematics	bool	Whether kinematic values are clipped to quantile bounds before fitting.
kinematics_winsorize_quantiles	tuple[float, float]	Lower and upper quantiles clipping kinematic values.
clamp_kinematics_range	bool	Kinematics normalization clamping - useful for datasets with very small action deltas.
min_kinematics_std	float	Minimum standard deviation before a kinematic dimension is treated as constant.
min_kinematics_range	float	Minimum value range before a kinematic dimension is treated as constant.
tokenization	TokenizationConfig	Tokenization.
color_augmentation	AugmentationPipelineConfig	Whether color augmentations are applied to camera images.
spatial_augmentation	AugmentationPipelineConfig | None	Whether spatial augmentations are applied to camera images.
skip_initial_episode_steps	int	: Whether to skip the initial n steps of each episode due to recording artifacts.
downsample_factor	int	: Whether to downsample each dataset episode by taking every n-th step.
action_backward_shift	int	: Number of steps to shift actions backward in the sequence, to compensate for hardware latency.
trailing_padded_actions	int | None	: Max trailing padding allowed per sampled window. Valid starts per episode: : episode_length - sequence_length + trailing_padded_actions + 1. If 0, only windows that fit : entirely within the episode are sampled (no right-side padding). When None (default), resolves : to pred_horizon - 1.
val_ratio	float	: Ratio of dataset episodes to use for validation.
total_ratio	float	: Ratio of total dataset episodes to use (for ablation studies on varying dataset sizes).
action_sample_size	int	: Number of action rows to stash on the normalizer per action key for downstream : data-aware initialization (e.g. mixture-density head k-means++). Set to 0 to : disable. Memory cost per action key is action_sample_size * action_dim * : bytes_per_element.