Skip to content

x86_inductor

x86_inductor

X86 Inductor backend for PT2E quantized operator lowering.

X86InductorBackend 

X86InductorBackend(is_dynamic=False, is_qat=False, reduce_range=False)

Bases: BasePT2EBackend

X86 Inductor backend for PT2E quantization and lowering.

Initialize X86 Inductor backend configuration.

Parameters:

Name Type Description Default
is_dynamic bool

Use dynamic activation quantization.

 False
is_qat bool

Use quantization-aware training observers.

 False
reduce_range bool

Reduce quantization range for older CPUs without VNNI.

 False
Source code in src/versatil/quantization/pt2e/backends/x86_inductor.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
def __init__(
    self,
    is_dynamic: bool = False,
    is_qat: bool = False,
    reduce_range: bool = False,
) -> None:
    """Initialize X86 Inductor backend configuration.

    Args:
        is_dynamic: Use dynamic activation quantization.
        is_qat: Use quantization-aware training observers.
        reduce_range: Reduce quantization range for older CPUs
            without VNNI.
    """
    self._is_dynamic = is_dynamic
    self._is_qat = is_qat
    self._reduce_range = reduce_range

name property 

name

Serialized PT2E backend name.

is_dynamic property 

is_dynamic

Whether this backend uses dynamic activation quantization.

is_qat property 

is_qat

Whether this backend uses QAT observer configuration.

supported_device_types property 

supported_device_types

X86 inductor only supports CPU inference.

create_quantizer 

create_quantizer(module_path)

Create an X86InductorQuantizer targeting a specific module.

Parameters:

Name Type Description Default
module_path str

Dotted path to the target submodule. Empty string means global (whole model).

 required

Returns:

Type Description
Quantizer

Configured X86InductorQuantizer.
Source code in src/versatil/quantization/pt2e/backends/x86_inductor.py 
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
def create_quantizer(self, module_path: str) -> Quantizer:
    """Create an X86InductorQuantizer targeting a specific module.

    Args:
        module_path: Dotted path to the target submodule.
            Empty string means global (whole model).

    Returns:
        Configured X86InductorQuantizer.
    """
    quantizer = X86InductorQuantizer()
    config = get_default_x86_inductor_quantization_config(
        is_dynamic=self._is_dynamic,
        is_qat=self._is_qat,
        reduce_range=self._reduce_range,
    )
    if module_path == "":
        quantizer.set_global(config)
    else:
        quantizer.set_module_name_qconfig(module_path, config)
    return quantizer

environment_context 

environment_context()

Context manager that sets and restores X86 Inductor env vars.

Sets CUDA_VISIBLE_DEVICES to empty (CPU-only), enables TORCHINDUCTOR_FREEZING, and enables cpp_wrapper. Restores original values on exit.

Yields:

Type Description
Generator[None]

None.
Source code in src/versatil/quantization/pt2e/backends/x86_inductor.py 
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
@contextmanager
def environment_context(self) -> Generator[None]:
    """Context manager that sets and restores X86 Inductor env vars.

    Sets CUDA_VISIBLE_DEVICES to empty (CPU-only), enables
    TORCHINDUCTOR_FREEZING, and enables cpp_wrapper. Restores
    original values on exit.

    Yields:
        None.
    """
    saved = {
        _CUDA_VISIBLE_DEVICES_KEY: os.environ.get(_CUDA_VISIBLE_DEVICES_KEY),
        _TORCHINDUCTOR_FREEZING_KEY: os.environ.get(_TORCHINDUCTOR_FREEZING_KEY),
    }
    saved_cpp_wrapper = inductor_config.cpp_wrapper
    os.environ[_CUDA_VISIBLE_DEVICES_KEY] = ""
    os.environ[_TORCHINDUCTOR_FREEZING_KEY] = "1"
    inductor_config.cpp_wrapper = True
    try:
        yield
    finally:
        for key, value in saved.items():
            if value is None:
                os.environ.pop(key, None)
            else:
                os.environ[key] = value
        inductor_config.cpp_wrapper = saved_cpp_wrapper

activate_environment 

activate_environment()

Set X86 Inductor env vars permanently.

Source code in src/versatil/quantization/pt2e/backends/x86_inductor.py 
113
114
115
116
117
def activate_environment(self) -> None:
    """Set X86 Inductor env vars permanently."""
    os.environ[_CUDA_VISIBLE_DEVICES_KEY] = ""
    os.environ[_TORCHINDUCTOR_FREEZING_KEY] = "1"
    inductor_config.cpp_wrapper = True