Xiaomi MiMo and TileRT Push 1-Trillion-Parameter Model Past 1000 Tokens per Second on Commodity GPUs
Inference speed becomes a competitive metric for large language models. Xiaomi’s MiMo team has just released MiMo-V2.5-Pro-UltraSpeed, developed in collaboration with the TileRT programming team. It records faster than 1000 tokens per second in a trillion parameter model. The Xiaomi team describes this as starting on a multi-billion parameter scale. The demos show a production … Read more