Skip to content

Conversation

@bayo-ibm
Copy link
Contributor

@bayo-ibm bayo-ibm commented Jun 13, 2025

Description of the change

This enables GPTQv2 for fms_mo

  1. Added new gptq arguments
  2. Changed the Readme file for the GPTQ example to reflect the changes

Related issues or PRs

NA

How to verify the PR

Was the PR tested

  • I have added >=1 unit test(s) for every new method I have added (if that coverage is difficult, please briefly explain the reason)
  • I have ensured all unit tests pass

Checklist for passing CI/CD:

  • All commits are signed showing "Signed-off-by: Name <email@domain.com>" with git commit -signoff or equivalent
  • PR title and commit messages adhere to Conventional Commits
  • Contribution is formatted with tox -e fix
  • Contribution passes linting with tox -e lint
  • Contribution passes spellcheck with tox -e spellcheck
  • Contribution passes all unit tests with tox -e unit

Note: CI/CD performs unit tests on multiple versions of Python from a fresh install. There may be differences with your local environment and the test environment.

@chichun-charlie-liu chichun-charlie-liu linked an issue Jun 23, 2025 that may be closed by this pull request
@chichun-charlie-liu
Copy link
Collaborator

in addition to adding 2 new args to trigger gptqv2, can we also add a simpler option, which allows a new quant_method, i.e. gptqv2, and the rest will be taken care automatically. feels like it would be more user-friendly this way...

@bayo-ibm bayo-ibm force-pushed the gptqv2 branch 3 times, most recently from 6003f4e to 9eed23b Compare July 16, 2025 06:05
Copy link
Collaborator

@chichun-charlie-liu chichun-charlie-liu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plz check pylint errors

Signed-off-by: omobayode.fagbohungbe <omobayode.fagbohungbe@ibm.com>
bayo-ibm added 2 commits July 17, 2025 11:17
Signed-off-by: omobayode.fagbohungbe <omobayode.fagbohungbe@ibm.com>
Signed-off-by: omobayode.fagbohungbe <omobayode.fagbohungbe@ibm.com>
@chichun-charlie-liu chichun-charlie-liu merged commit 008e0e3 into foundation-model-stack:main Jul 17, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate from autoGPTQ to GPTQModel

4 participants