chore(deps): Bump torchao from 0.11 to 0.12.0

[dependabot]

Bumps torchao from 0.11 to 0.12.0.

Release notes

Sourced from torchao's releases.

v0.12.0

Highlights

We are excited to announce the 0.12.0 release of torchao! This release adds support for QAT + Axolotl Integration and prototype MXFP/NVFP support on Blackwell GPUs!

QAT + Axolotl Integration

TorchAO’s QAT support has been integrated into Axolotl’s fine-tuning recipes! Check out the docs here or run it yourself using the following command:

axolotl train examples/llama-3/3b-qat-fsdp2.yaml
axolotl quantize examples/llama-3/3b-qat-fsdp2.yaml

Initial results for Llama3.2-3B by @​SalmanMohammadi (axolotl-ai-cloud/axolotl#2590):

Model/Metric	hellaswag acc	hellaswag acc_norm	wikitext bits_per_byte	wikitext byte_perplexity	wikitext word_perplexity
bfloat16	0.5552	0.7315	0.6410	1.5594	10.7591
bfloat16 PTQ	0.5393	0.7157	0.6613	1.5815	11.6033
qat ptq	0.5423	0.7180	0.6567	1.5764	11.4043
Recovered (qat ptq)	18.87%	14.56%	22.66%	23.08%	23.57%

[Prototype | API not finalized] MXFP and NVFP support on Blackwell GPUs

TorchAO now includes prototype support for NVFP4 (NVIDIA's 4-bit floating-point format) and Microscaling (MX) formats on NVIDIA's latest Blackwell GPU architecture. These formats enable efficient inference, achieving up to 61% end-to-end performance improvement in vLLM on Qwen3 models and near 2x speedups for diffusion workloads.

To use:

from torchao.quantization import quantize_ 
from torchao.prototype.mx_formats import (
    MXFPInferenceConfig,
    NVFP4InferenceConfig,
)
# Quantize model with MXFP8 
model = quantize_(model, MXFPInferenceConfig(block_size=32))
# Quantize model to NVFP4 (without double scaling)
model = quantize_(model, NVFP4InferenceConfig())

Note: This is a prototype feature with APIs subject to change. Requires NVIDIA Blackwell GPUs (B200, 5090) with CUDA 12.8+.

BC Breaking

• Remove preserve_zero and zero_point_domain from choose_qparams_affine (pytorch/ao#2149)
• Rename qparams for tinygemm (pytorch/ao#2344)
• Convert quant_primitives methods private (pytorch/ao#2350)
• Delete Galore (pytorch/ao#2397)
• Remove more Galore bits (pytorch/ao#2417)

... (truncated)

Commits

• 442232f Update workflows to use torchao-release0.12 test-infra branch
• fe99758 Fix tutorials (#2516)
• ddd4021 Revert "Update function params and corresponding usages." (#2523)
• 20d4503 Update function params and corresponding usages. (#2522)
• d9f8a68 Add offset function for activation and weight for multithread. (#2514) (#2521)
• f24f37b [CPU] Add concat-linear fusion pass for da8w4 (#2476)
• a45b1f7 [CPU INT8 SDPA] use manual transpose and pack (#2380)
• 64c1ce3 Tutorial for benchmarking (#2499)
• ee38153 Fix docstrings for quantization API docs (#2471)
• 19c009d TorchAO new observers (#2508)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot]

Superseded by #183.