Introduction
In the ever-evolving world of artificial intelligence and gaming, getting the right hardware mix is crucial to maximize performance. Today, we will explore how to combine an RTX 5080 with an RTX 3090 to reach impressive speeds of 80 tokens per second on the Qwen 3.6 27B model with Q8 quantization.
Why Combine the RTX 5080 and RTX 3090?
When looking to boost your AI applications, every hardware detail counts. The RTX 5080, with its raw power, and the RTX 3090, with its large 24GB memory, form an ideal combination for local LLM (Large Language Models) experiments. Together, they offer a perfect balance of computational power and memory management.
Setup Basics
To achieve the desired performance, it is essential to have a motherboard capable of managing these two graphics cards. The Asus Prime X570-Pro is recommended as it allows the PCIe slot to be split into 2x8, essential for the simultaneous operation of both GPUs.
BIOS Configuration
BIOS setup is a crucial step. Here are the settings to adjust:
- Disable the 'CSM' (Compatibility Support Module) to allow the use of both cards.
- Enable 'Above 4G Decoding'.
- Set 'ReSize BAR Support' to Auto or Enabled.
- Ensure that PCIEX16_1 and PCIEX16_2 Link Modes are set to Gen 4.
Driver Installation
Installing drivers for two different GPU models can be tricky. Use the standard NVIDIA driver if you own cards from different generations. Ensure a clean installation and verify card recognition with nvidia-smi.
Achieving 80 Tok/s Performance
With the right configuration, achieving 80 tok/s on Qwen 3.6 27B Q8 becomes a reality. This model, with its Q8 quantization, greatly benefits from the combined power of the two cards. Tests have shown that this setup could even exceed these figures with further optimizations.
Conclusion
By combining the RTX 5080 and RTX 3090, you can significantly enhance your AI capabilities for complex tasks. However, it is crucial to carefully follow each configuration step to ensure optimal operation. Ready to take action?
Let's discuss your project in 15 minutes.