I'm a bit concerned the build uses a Gold Certified power supply unit?
Even for cheaper builds for non-ML workstations I would still only use Platinum and nothing less. I've been told Titanium is excessive but I mean I leave these things on for a while and power is expensive.
For the DIY enthusiast or the WFH researcher, also the amount of heat involved can be a considerable cost in cooling or utility cost which varies substantially by floor of a building. It's probably not good, but not that bad to aircool this many GPUs as I've done in the past but it definitely means I'm paying a lot for A/C in the summer but almost nothing in the winter.
I think Smerity even said he heated his small bedroom through the San Francisco winter off of one GPU while researching YOLO.
Point: These things get hot. They require a lot of electricity. You should be concerned about a good PSU even for smaller builds. My energy cost for a 6GPU rig ran me about 1/3 of my total rent for a small apartment. That's electricity BEFORE I calculated my A/C bill which was separate and also substantial. My landlord hates me because I initially talked him into including it with my rent.
All in all, it still makes sense to keep investing in local workstations, on-premises builds. No security concerns about a cloud, no futzing around with integrated notebooks, you own it you control it, and the price point up front is extremely attractive compared to base rates for cloud computing even on specialized hardware like a TPU.
The numbers I come up with for batches still have a wide gap of several thousand USD most of the time, and then there's how much time it takes and how likely their service breaks.
So kudos for the person who put in the effort to put this together and share. Any and all efforts towards making ML/DS affordable and DIY rises the tide for all boats.
Question to the audience: Does anyone build GPU rigs like this for cryptocurrency anymore? I was only able to build a workstation once the price for GPU cards crashed.
The efficiency delta between Gold and Titanium is really small. Optimizing that for heat reasons would be optimizing less than 1% of total heat output. And most cases keep the power supply thermally separate from the rest of the stuff anyway.
This guy has a very oversized Gold power supply, the efficiency would be ~92% with gold vs 94% with platinum. Maybe a smaller titanium one would be a better overall choice I guess.
Generally I think going jumbo is good for cooling, because honestly the weight for this build probably doesn't matter as I imagine it's not getting transported often.
Conversely, this is not a build for overclocking per se. However I think it's a safe assumption we are running at capacity of over 90% for multiple days or weeks even. If batches run into over a month, probably time to get a server rack instead?
It is worthwhile to note you're not going to save any money in efficiency for computation you don't use.
Not sure about the latest PSUs, but in the past high efficiency PSU's had to make trade offs with increased ripple/noise so they weren't ideal where maximum stability was desired.
Quick approximation: maybe $10 if it really is 92% to 94% efficiency? (at a low $0.10/kw). Although other commenters say it is a bigger efficiency difference.
Even for cheaper builds for non-ML workstations I would still only use Platinum and nothing less. I've been told Titanium is excessive but I mean I leave these things on for a while and power is expensive.
For the DIY enthusiast or the WFH researcher, also the amount of heat involved can be a considerable cost in cooling or utility cost which varies substantially by floor of a building. It's probably not good, but not that bad to aircool this many GPUs as I've done in the past but it definitely means I'm paying a lot for A/C in the summer but almost nothing in the winter.
I think Smerity even said he heated his small bedroom through the San Francisco winter off of one GPU while researching YOLO.
Point: These things get hot. They require a lot of electricity. You should be concerned about a good PSU even for smaller builds. My energy cost for a 6GPU rig ran me about 1/3 of my total rent for a small apartment. That's electricity BEFORE I calculated my A/C bill which was separate and also substantial. My landlord hates me because I initially talked him into including it with my rent.
All in all, it still makes sense to keep investing in local workstations, on-premises builds. No security concerns about a cloud, no futzing around with integrated notebooks, you own it you control it, and the price point up front is extremely attractive compared to base rates for cloud computing even on specialized hardware like a TPU.
The numbers I come up with for batches still have a wide gap of several thousand USD most of the time, and then there's how much time it takes and how likely their service breaks.
So kudos for the person who put in the effort to put this together and share. Any and all efforts towards making ML/DS affordable and DIY rises the tide for all boats.
Question to the audience: Does anyone build GPU rigs like this for cryptocurrency anymore? I was only able to build a workstation once the price for GPU cards crashed.