Skip to content

Known Issues

Compatibility patches for upstream bugs in torchao.

torchao 0.17 + Python 3.14: PT2E import crash

Error: TypeError: cannot set '__module__' attribute of immutable type 'types.UnionType' when importing torchao.quantization.pt2e.

Root cause: Python 3.14 makes typing.Union objects immutable. torchao 0.17 still assigns __module__ to Union aliases at module import time.

Upstream:

Workaround: src/versatil/quantization/torch_patches.py (register_torchao_patches) installs an import hook that rewrites the crashing __module__ assignments to pass in memory as the torchao modules load. Registered automatically when versatil is imported.

Remove when: a clean torchao wheel imports torchao.quantization.pt2e on Python 3.14 without the patch.

torchao 0.17: QAT INT4 group size is ignored

Error: RuntimeError: shape '[..., -1, 128]' is invalid during a QAT training forward prepared from Int4WeightOnlyConfig(group_size=32) or group_size=64.

Root cause: torchao 0.17 infers Int4WeightFakeQuantizeConfig(group_size=128) for Int4WeightOnlyConfig(version=2), ignoring the user-provided Int4WeightOnlyConfig.group_size.

Upstream:

Workaround: src/versatil/quantization/torch_patches.py (register_torchao_patches) installs an import hook that rewrites the QAT fake-quant config in memory as the module loads, so it uses base_config.group_size. Wheels that already include the upstream fix are accepted unchanged; unknown torchao source raises instead of silently misquantizing.

Remove when: the pinned torchao wheel includes the fix from pytorch/ao#4518.