Tag: inference-optimization
What One AI Query Costs: Tokens, Bytes, Joules, Cents
One question, one model, one machine — costed end to end. Then four dials that move the same answer by more than 10,000x.
Tencent HPC-Ops: SOTA Performance for LLM Inference
Tencent releases HPC-Ops, a production-grade high-performance operator library for LLM inference, delivering up to 2.22x speedup on NVIDIA H20 GPUs.
NVIDIA TTT-E2E: Test-Time Training Long Context
NVIDIA researchers introduce TTT-E2E, a test-time training approach that enables models to handle ultra-long contexts efficiently through dynamic adaptation.