latent

latent

Latent action posterior and prior network configurations.

PosteriorLatentEncoderConfig dataclass

PosteriorLatentEncoderConfig(_target_=MISSING, latent_dimension=MISSING, device='${policy.device}')

Base posterior encoder configuration.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of the latent variable.
device	str	Torch device for the module.

PriorLatentEncoderConfig dataclass

PriorLatentEncoderConfig(_target_=MISSING, latent_dimension=MISSING, device='${policy.device}')

Base latent prior configuration.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of latent variable z.
device	str	Device to place prior on.

VAETransformerEncoderConfig dataclass

VAETransformerEncoderConfig(_target_='versatil.models.decoding.latent.posterior.transformer_encoder.VAETransformerEncoder', latent_dimension=MISSING, device='${policy.device}', embedding_dimension=MISSING, prediction_horizon='${policy.prediction_horizon}', observation_horizon='${policy.observation_horizon}', number_of_heads=8, feedforward_dimension=512, number_of_encoder_layers=4, activation=value, dropout_rate=0.1, attention_dropout=0.0, normalization_type=value, attention_type=value, positional_encoding_type=None, exclude_keys=None, min_logvar=None, deterministic=False, mu_tanh_bound=None, max_logvar=None)

Bases: PosteriorLatentEncoderConfig

Transformer-based VAE latent action encoder configuration.

This encoder uses a transformer architecture to encode action sequences into a latent space via variational inference.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of VAE latent space, i.e. the dimension of the output z.
embedding_dimension	int	Dimension of the output embedding.
prediction_horizon	int	Number of action timesteps.
observation_horizon	int	Number of observation timesteps.
device	str	Device to place encoder on.
number_of_heads	int	Number of attention heads.
feedforward_dimension	int	Feedforward network dimension.
number_of_encoder_layers	int	Number of transformer encoder layers.
activation	str	Activation function name.
dropout_rate	float	Dropout probability.
attention_dropout	float	Dropout probability inside attention.
normalization_type	str	Normalization layer type.
attention_type	str	Attention mechanism type (use AttentionType enum values).
positional_encoding_type	str | None	Self-attention positional encoding type.
exclude_keys	list[str] | None	List of keys to exclude from encoding.
min_logvar	float | None	Minimum log variance for avoiding variance collapse.
deterministic	bool	If True, output deterministic embeddings without reparameterization. Use with MMD or OT regularizers instead of KL divergence.
mu_tanh_bound	float | None	Optional symmetric bound for posterior mu. When set, applies bound * tanh(raw_mu / bound) before sampling/returning z.
max_logvar	float | None	Optional maximum log variance for avoiding variance explosion.

GaussianPriorConfig dataclass

GaussianPriorConfig(_target_='versatil.models.decoding.latent.prior.gaussian_prior.GaussianPrior', latent_dimension=32, device='${policy.device}', infer_constant_prior=False)

Bases: PriorLatentEncoderConfig

Standard Gaussian N(0, I) prior configuration.

Simple non-learned prior that samples from a standard normal distribution. This is the default prior for variational algorithms when no learned prior is specified.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of latent variable z.
infer_constant_prior	bool	ACT-style constant zero latent at inference instead of N(0, I) samples.

PriorTransformerEncoderConfig dataclass

PriorTransformerEncoderConfig(_target_='versatil.models.decoding.latent.prior.transformer_encoder.PriorTransformerEncoder', latent_dimension=MISSING, device='${policy.device}', embedding_dimension=MISSING, prediction_horizon='${policy.prediction_horizon}', observation_horizon='${policy.observation_horizon}', number_of_heads=8, feedforward_dimension=512, number_of_encoder_layers=4, activation=value, dropout_rate=0.1, attention_dropout=0.0, normalization_type=value, attention_type=value, positional_encoding_type=None, exclude_keys=None, learn_variance=True, min_logvar=None, deterministic=False, max_logvar=None)

Bases: PriorLatentEncoderConfig

Configuration for the transformer-based prior latent encoder.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of the latent variable.
embedding_dimension	int	Embedding dimension of the model tokens.
prediction_horizon	int	Number of future actions predicted per chunk.
observation_horizon	int	Number of past observation frames consumed.
device	str	Torch device for the module.
number_of_heads	int	Attention head count.
feedforward_dimension	int	Feedforward layer width.
number_of_encoder_layers	int	Transformer encoder layer count.
activation	str	Activation function name.
dropout_rate	float	Dropout probability.
attention_dropout	float	Dropout probability inside attention.
normalization_type	str	Normalization layer type.
attention_type	str	Attention implementation name.
positional_encoding_type	str | None	Self-attention positional encoding type.
exclude_keys	list[str] | None	Feature keys excluded from prior conditioning.
learn_variance	bool	Whether the prior variance is learned instead of fixed.
min_logvar	float | None	Lower clamp for the learned log-variance.
deterministic	bool	Whether sampling returns the mean instead of drawing noise.
max_logvar	float | None	Upper clamp for the learned log-variance.

VampPriorConfig dataclass

VampPriorConfig(_target_='versatil.models.decoding.latent.prior.vamp_prior.VampPrior', latent_dimension=32, device='${policy.device}', num_components=50, action_space='${policy.action_space}', prediction_horizon='${policy.prediction_horizon}', min_logvar=None)

Bases: PriorLatentEncoderConfig

VampPrior (Variational Mixture of Posteriors) configuration.

Reference: "VAE with a VampPrior" (Tomczak & Welling, 2018)

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of latent variable z.
num_components	int	Number of mixture components K.
action_space	ActionSpaceConfig	ActionSpace defining the action dimensions.
prediction_horizon	int	Number of timesteps in action chunks.
min_logvar	float | None	Optional minimum logvar clamp.

DiTPriorConfig dataclass

DiTPriorConfig(_target_='versatil.models.decoding.latent.prior.dit_prior.DiTPrior', latent_dimension=32, device='${policy.device}', embedding_dimension=256, number_of_heads=8, number_of_layers=4, feedforward_dimension=1024, observation_horizon='${policy.observation_horizon}', algorithm_type=value, sigma=0.0, ode_solver=value, timestep_sampler=BETA.value, logit_mean=0.0, logit_std=1.0, beta_alpha=1.5, beta_beta=1.0, max_timestep=0.999, num_train_timesteps=100, num_inference_steps=10, beta_start=0.0001, beta_end=0.02, beta_schedule=value, scheduler_type=value, prediction_type=value, clip_sample=False, variance_type=None, dropout=0.1, normalization_type=value, attention_type=value, number_of_key_value_heads=None, activation=value, use_gating=True, exclude_keys=None, prior_target_key=value, latent_standardization_enabled=True, latent_standardization_eps=1e-06, latent_standardization_max_batches=None, require_fitted_latent_standardization=False)

Bases: PriorLatentEncoderConfig

DiT-style transformer prior for denoising score matching.

Uses a non-autoregressive diffusion transformer where noisy latent z is treated as a CLS token appended to observation tokens.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of latent variable z.
embedding_dimension	int	Hidden dimension of the transformer.
number_of_heads	int	Number of attention heads.
number_of_layers	int	Number of DiT decoder layers.
feedforward_dimension	int	Dimension of the feedforward network.
observation_horizon	int	Observation history size.
algorithm_type	str	Algorithm type ("diffusion" or "flow_matching").
sigma	float	Noise level for flow matching (0 = deterministic OT).
ode_solver	str	ODE solver for flow matching ("euler", "heun", or "rk4").
timestep_sampler	str	Distribution the diffusion timestep is drawn from.
logit_mean	float	Mean of the logit-normal timestep sampler.
logit_std	float	Standard deviation of the logit-normal timestep sampler.
beta_alpha	float	Alpha parameter of the beta timestep sampler.
beta_beta	float	Beta parameter of the beta timestep sampler.
max_timestep	float	Largest sampled diffusion timestep.
num_train_timesteps	int	Number of diffusion timesteps during training.
num_inference_steps	int	Number of denoising/integration steps.
beta_start	float	Starting beta for noise schedule (diffusion).
beta_end	float	Ending beta for noise schedule (diffusion).
beta_schedule	str	Type of noise schedule (diffusion).
scheduler_type	str	Diffusion scheduler type.
prediction_type	str	What diffusion model predicts (epsilon, sample, velocity).
clip_sample	bool	Whether to clip samples during diffusion.
variance_type	str | None	Variance type for DDPM scheduler.
dropout	float	Dropout rate.
normalization_type	str	Type of adaptive normalization layer.
attention_type	str	Attention implementation name.
number_of_key_value_heads	int | None	Key/value head count for grouped-query attention.
activation	str	Activation function name.
use_gating	bool	Whether to use AdaLN-Zero gating in DiT layers.
exclude_keys	list[str] | None	Keys to exclude from observations.
prior_target_key	str	Posterior output key used as denoising target.
latent_standardization_enabled	bool	Whether to standardize DiT target latents.
latent_standardization_eps	float	Numerical epsilon used in latent standardization.
latent_standardization_max_batches	int | None	Maximum train batches to scan when fitting latent standardization stats. None scans the full train loader.
require_fitted_latent_standardization	bool	Whether missing latent stats should raise.

VQPosteriorEncoderConfig dataclass

VQPosteriorEncoderConfig(_target_='versatil.models.decoding.latent.posterior.vq_encoder.VQPosteriorEncoder', latent_dimension=8, device='${policy.device}', num_codes=4, num_residual_layers=1, embedding_dimension=64, prediction_horizon='${policy.prediction_horizon}', observation_horizon='${policy.observation_horizon}', ema_decay=0.99, dead_code_threshold=1.0, number_of_heads=4, feedforward_dimension=128, number_of_encoder_layers=1, activation=value, dropout_rate=0.0, attention_dropout=0.0, normalization_type=value, attention_type=value, positional_encoding_type=None, exclude_keys=None)

Bases: PosteriorLatentEncoderConfig

VQ posterior encoder configuration.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of each codebook vector and the latent space passed to the decoder.
num_codes	int	Number of codebook entries per residual layer (K).
num_residual_layers	int	Number of cascading VQ layers.
embedding_dimension	int	Transformer hidden dimension.
prediction_horizon	int	Number of action timesteps.
observation_horizon	int	Number of observation timesteps.
device	str	Device string.
ema_decay	float	EMA decay for codebook updates.
dead_code_threshold	float	Cluster size below which codes are replaced.
number_of_heads	int	Number of attention heads.
feedforward_dimension	int	Feedforward network dimension.
number_of_encoder_layers	int	Number of transformer encoder layers.
activation	str	Activation function name.
dropout_rate	float	Dropout probability.
attention_dropout	float	Dropout probability inside attention.
normalization_type	str	Normalization layer type.
attention_type	str	Attention mechanism type (use AttentionType enum values).
positional_encoding_type	str | None	Self-attention positional encoding type.
exclude_keys	list[str] | None	Observation keys to exclude from encoding.

UniformCodebookPriorConfig dataclass

UniformCodebookPriorConfig(_target_='versatil.models.decoding.latent.prior.uniform_codebook_prior.UniformCodebookPrior', latent_dimension=8, device='${policy.device}', num_codes=4, num_residual_layers=1)

Bases: PriorLatentEncoderConfig

Uniform categorical prior over VQ codebook indices.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of each codebook vector.
num_codes	int	Number of codebook entries per layer (K).
num_residual_layers	int	Number of residual VQ layers.

CodebookPriorConfig dataclass

CodebookPriorConfig(_target_='versatil.models.decoding.latent.prior.codebook_prior.CodebookPrior', latent_dimension=8, device='${policy.device}', num_codes=4, num_residual_layers=1, embedding_dimension=64, observation_horizon='${policy.observation_horizon}', number_of_heads=4, feedforward_dimension=128, number_of_encoder_layers=1, activation=value, dropout_rate=0.0, attention_dropout=0.0, normalization_type=value, attention_type=value, positional_encoding_type=None, exclude_keys=None, temperature=1.0)

Bases: PriorLatentEncoderConfig

Learned categorical prior over VQ codebook indices.

Attributes:

Name	Type	Description
_target_	str	Import path instantiated by Hydra.
latent_dimension	int	Dimension of each codebook vector. Must match the posterior encoder's latent dimension.
num_codes	int	Number of codebook entries per layer (K).
num_residual_layers	int	Number of residual VQ layers.
embedding_dimension	int	Transformer hidden dimension.
observation_horizon	int	Number of observation timesteps.
device	str	Device string.
number_of_heads	int	Number of attention heads.
feedforward_dimension	int	Feedforward network dimension.
number_of_encoder_layers	int	Number of transformer encoder layers.
activation	str	Activation function name.
dropout_rate	float	Dropout probability.
attention_dropout	float	Dropout probability inside attention.
normalization_type	str	Normalization layer type.
attention_type	str	Attention mechanism type (use AttentionType enum values).
positional_encoding_type	str | None	Self-attention positional encoding type.
exclude_keys	list[str] | None	Observation keys to exclude from encoding.
temperature	float	Softmax temperature for sampling. Lower values produce sharper categorical distributions.