Introduction
In 2024, I quit my FAANG job to become an independent researcher. To do this, I needed powerful GPU resources, so I built "grumbl," a server housing 6 GPU 6000 Ada, costing a total of $48,000. The crucial question was: Was this investment worth it, or should I have rented cloud GPUs?
Why Build a GPU Server?
At first glance, $48,000 may seem exorbitant for a GPU server. However, as a researcher, my time is invaluable. If more powerful GPUs could accelerate my research by just two months, the investment is justified. Every month gained can offset the income loss from quitting my previous job.
Choosing the GPUs
My choice fell on the 6000 Ada based on Tim Dettmers' guide. The A100s don't support FP8 and have slower inference performance. As for the H100s, their cost didn't justify the investment compared to the 6000 Ada for my specific needs.
Electrical Constraints and Solutions
Living in an apartment, upgrading the electrical circuits for standard datacenter servers wasn't an option. I opted for two power supplies and plugged them into two outlets on separate circuits. This setup is risky if done incorrectly, but I hired a professional to ensure safety. Ironically, after all these precautions, I ended up moving grumbl to my parents' basement, where I could upgrade the circuits.
Comparison with the Cloud
I calculated that in 2024, at current GPU rental rates, it would take about a year of 85% utilization to match the cost of renting in the cloud. I wrote a script to log each GPU's usage every minute and the power consumption in watts to calculate electricity costs.
Cost Analysis and ROI
The analysis showed that even with electricity costs, my server was more cost-effective than cloud rental after a year of intensive use. Additionally, owning the hardware allowed me to avoid unexpected cloud rate hikes and enjoy total flexibility.
Conclusion
So, was my $48K GPU server worth it? Yes, for my specific use case. It accelerated my research and gave me complete control over my hardware. However, every project is unique, and it's crucial to thoroughly assess needs before investing.
Let's discuss your project in 15 minutes.