Expert-parallelism communication library for mixture-of-experts training that balances load with dynamic redundant experts while efficiently optimizing collective communication operations and memory layout.