Tag: Scaling Laws
Training vs Inference: Two Economies, Two Opposite Bottlenecks
Training is network-bound and paid once. Serving is memory-bound and paid forever. The crossover point, and why the 2024 chip-demand call was backwards.
ATLAS: New Scaling Laws for Multilingual AI Models
Google DeepMind's Atlas research explores multilingual scaling laws, providing a framework for training highly efficient models across hundreds of languages.