Skip to content

dynamic_feature_embedding

dynamic_feature_embedding

DynamicFeatureEmbedding 

DynamicFeatureEmbedding(embedding_dimension)

Bases: ModuleAttrMixin

Learned embeddings for features, created on-demand at runtime.

This module uses lazy initialization - embeddings are created on first access. To support loading checkpoints, it overrides _load_from_state_dict to create embeddings dynamically from the state dict.

Source code in src/versatil/models/layers/dynamic_feature_embedding.py 
15
16
17
18
def __init__(self, embedding_dimension: int):
    super().__init__()
    self.embedding_dimension = embedding_dimension
    self.embeddings = nn.ParameterDict()

forward 

forward(name, device)

Get or create a learned embedding for the given feature name.

Source code in src/versatil/models/layers/dynamic_feature_embedding.py 
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
def forward(self, name: str, device: torch.device) -> torch.Tensor:
    """Get or create a learned embedding for the given feature name."""
    key = name.replace(".", "_")
    if key not in self.embeddings:
        self.embeddings[key] = nn.Parameter(
            torch.randn(
                1,
                1,
                self.embedding_dimension,
                device=device,
                dtype=self.dtype,
            )
            * 0.02
        )
    return self.embeddings[key]