Building a Powerful Desktop with Multi-GPU Video Card Connectivity:Fear Factor of NVIDIA K80s

[Isaac]

Hello, I'm recently getting into training machine learning models. But as I progress my i7 12th gen CPU is not really enough. Google Colab has been really great for some of my projects, but it too has a ram limitation and that's after paying for their pro version. I'm probably going to just rent a hosted VM and use that for my bigger ML projects, because it's more bang for you buck.

However... I would like to have my own system that can do what I need:

I'm hoping to fit 4 Nvidia K80s on one motherboard. I assume this means I need a built-in gpu. This also means I'll need to break my ML models into parts, which sounds scary as well. So it's possible I'll be building something that's not even going to work correctly after I build it.

The idea is you have 12 GB of VRAM x 8 GPUS. This means each K80 is 2 cards.

K80s have passive cooling, which is a new thing for me.

Does anyone have any background in building something like this? I've seen other builds online, but nothing seems to be using 4 of these things. Maybe my approach is wrong?

[fliptw]

this is the kind of thing server racks are made for.

[Isaac]

this is the kind of thing server racks are made for.

Agreed and I'm not super happy about the work arounds people are doing for passive cooling. I've seen one guy remove the heat sink and add a water cooling system. Yes it's very interesting, but I'd rather it just work as intended