PyTorch Monarch: Distributed Programming
PyTorch announces Project Monarch, a new compiler backend that significantly improves training efficiency for Mixture-of-Experts (MoE) models on NVIDIA GPUs.