Introduction
In a world where inference demands are booming, the ability to efficiently and economically handle complex tasks is crucial. In this context, Wafer, in collaboration with Vercel AI Gateway and OpenRouter, has announced a major breakthrough with the GLM5.2 model running on the AMD MI355X GPU. This combination offers a performance of 2626 tokens per second per node while reducing costs by more than two times compared to the Blackwell solution.
The Challenge of Cost-Effective Performance
As AI models like Claude Fable, GLM5.2, and Minimax M3 become standard, the demand for efficient inference solutions is increasing. However, the supply of high-end GPUs like those from NVIDIA isn't keeping up. Prices for NVIDIA GPUs, especially the Blackwell, are skyrocketing, making inference more expensive than ever.
The Competitive Edge of AMD
AMD, with its MI355X GPU, offers an economical and high-performance alternative. On average, AMD GPUs are 2.75 times cheaper than their NVIDIA counterparts, while offering comparable hardware specifications. Although NVIDIA's software ecosystem gives them an initial advantage, AMD is quickly closing the gap through innovative optimizations.
Demonstration of Performance
Wafer successfully achieved an aggregate throughput of 2626 tokens per second per node, with a response time of less than 5 seconds, on a workload of 20k inputs / 1k outputs. Although the measured performance is only 80% of a B200, the cost is more than twice as low.
Technical Details
To achieve these performances, Wafer used a lossless MXFP4 quantization, surpassing the official FP8 quantization. This quantization process enabled unprecedented processing efficiency.
Implications for the Future
This advancement shows that it is possible to push the boundaries of performance while keeping costs under control. For companies seeking scalable and affordable inference solutions, the AMD option presents itself as a credible and competitive alternative.
Conclusion
The combination of the GLM5.2 model with the AMD MI355X GPU opens new perspectives for large-scale inference processing. In addition to reducing costs, this solution allows for maintaining high performance, which is crucial for today's tech companies.
Let's discuss your project in 15 minutes.