Repository navigation

[feat] Tensor parallelism for MiniMax-H3 - #14609

Open

JingyaHuang wants to merge 12 commits into

huggingface:mainfrom

JingyaHuang:add-h3-tp-support

Commits on Oct 9, 2026
- Add tensor-parallel support for MiniMax-H3
  JingyaHuang
  committed
  764e7c8
- Shard MiniMax-H3's adaln_proj to fit tensor parallelism on one device
  
  JingyaHuang
  and
  claude
  committed
  58d9b51
- Build MiniMax-H3's row timestep plan on CPU, as its caller expects
  
  JingyaHuang
  and
  claude
  committed
  c766b88
- feat:combine tp+cp
  JingyaHuang
  committed
  4162784
- Allow tensor parallelism and context parallelism in one ParallelConfig
  
  authored and
  JingyaHuang
  committed
  7f649f9
- feat: validate the support on TPU
  JingyaHuang
  committed
  673a3f1
- fix: put tp first in the combined TP+CP mesh
  JingyaHuang
  committed
  3db653d
- fix: style
  JingyaHuang
  committed
  9c0f7aa
- fix: apply context parallelism on the sharded-load path
  JingyaHuang
  committed
  bbc55a7
- revert: attention head padding
  JingyaHuang
  committed
  6ce34c7
Commits on Oct 10, 2026
- revert: drop the Neuron all-gather padding and restore the TPU pre-shard path
  JingyaHuang
  committed
  986ea9a
- refactor: share the CPU pre-shard path between Neuron and TPU tensor parallelism
  JingyaHuang
  committed
  6845f33