Skip to content

Conversation

@ani300
Copy link
Contributor

@ani300 ani300 commented Jul 28, 2025

Description of the change

This PR add version limits for torchao, and fixes compatibility for AIU + the new 0.12 version.

Related issues or PRs

Issue in internal Spyre tracker.

How to verify the PR

When tracing an FX graph with activations that are dynamically casted to FP8, you should see the following decomposition for the _quantize_affine_float8 function:

         # File: /tmp/devel/src/venv/lib64/python3.12/site-packages/torchao/dtypes/affine_quantized_tensor.py:468 in from_hp_to_floatx, code: data = _quantize_affine_float8(input_float, scale, target_dtype)
        _to_copy_8: "f32[1, 128, 4096][524288, 4096, 1]cpu" = torch.ops.aten._to_copy.default(mul_2, dtype = torch.float32)
        unsqueeze_3: "f32[1, 128, 1, 1][128, 1, 1, 1]cpu" = torch.ops.aten.unsqueeze.default(_to_copy_7, 3)
        expand_1: "f32[1, 128, 1, 4096][128, 1, 1, 0]cpu" = torch.ops.aten.expand.default(unsqueeze_3, [1, 128, 1, 4096]);  unsqueeze_3 = None
        clone_1: "f32[1, 128, 1, 4096][524288, 4096, 4096, 1]cpu" = torch.ops.aten.clone.default(expand_1, memory_format = torch.contiguous_format);  expand_1 = None
        view_8: "f32[1, 128, 4096][524288, 4096, 1]cpu" = torch.ops.aten.view.default(clone_1, [1, 128, 4096]);  clone_1 = None
        div_3: "f32[1, 128, 4096][524288, 4096, 1]cpu" = torch.ops.aten.div.Tensor(_to_copy_8, view_8);  _to_copy_8 = view_8 = None
        clamp_1: "f32[1, 128, 4096][524288, 4096, 1]cpu" = torch.ops.aten.clamp.default(div_3, -448.0, 448.0);  div_3 = None
        _to_copy_9: "f8e4m3fn[1, 128, 4096][524288, 4096, 1]cpu" = torch.ops.aten._to_copy.default(clamp_1, dtype = torch.float8_e4m3fn);  clamp_1 = None

Was the PR tested

  • I have added >=1 unit test(s) for every new method I have added (if that coverage is difficult, please briefly explain the reason)
  • I have ensured all unit tests pass

Checklist for passing CI/CD:

  • All commits are signed showing "Signed-off-by: Name <email@domain.com>" with git commit -signoff or equivalent
  • PR title and commit messages adhere to Conventional Commits
  • Contribution is formatted with tox -e fix
  • Contribution passes linting with tox -e lint
  • Contribution passes spellcheck with tox -e spellcheck
  • Contribution passes all unit tests with tox -e unit

Note: CI/CD performs unit tests on multiple versions of Python from a fresh install. There may be differences with your local environment and the test environment.

Signed-off-by: Antoni Viros i Martin <aviros@ibm.com>
@ani300 ani300 changed the title Add version limits for torchao, ensure compat with 0.12 + AIU fix: Add version limits for torchao, ensure compat with 0.12 + AIU Jul 28, 2025
@github-actions github-actions bot added the fix label Jul 28, 2025
Copy link
Collaborator

@andrea-fasoli andrea-fasoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixes compatibility issue between torchao 0.12 and AIU.
Looks good to me

@andrea-fasoli andrea-fasoli merged commit f9ca98a into main Jul 29, 2025
15 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants