Repository navigation
[Bugfix] Fix Cosmos3 FP8 tensor parallelism - #14921
Conversation
|
Hi @yzhautouskay, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice. Once the PR links an issue (or gets the |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
What does this PR do?
Skip ModelOpt quantizer children during row-wise sharding to fix:
AttributeError: 'TensorQuantizer' object has no attribute 'weight'Tested with nvidia/Cosmos3-Super, revision fp8, using mixed W8A16/W8A8 inference on GB200: TP2/CP2 and TP4/CP1, 720p, 189 frames, 35 steps.
tp_fp8.mp4
Before submitting
self-reviewskill on the diff?documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.